We've had sparkline charts of response time for a long time on our Endpoints tab, and now have extended them to the throughput column as well. Quickly isolate endpoints that are spiking with traffic, and drill down into them to diagnose. No need to hunt to see what's causing an unusual trend in traffic.
As reported on Issue #228, if scout_apm is disabled on a node via the configuration monitor = false, we don't intend to install any instruments, but a few snuck in anyway. Since the rest of the agent isn't running, they (slowly but steadily) built up recorded info, but didn't purge it, causing a slow memory leak that became clear over the course of a week or two. We've stopped the offending instruments from installing themselves when Scout is disabled.
Recently the Ecto library released a major update - Version 3.0. As part of the changes, some of the hooks that Scout relied on to capture data have changed. No longer can we hook into Ecto's logging system to extract information about queries being run, and instead we have a proper Telemetry event to listen to.
The GitHub Readme describes Falcon as, "... *a multi-process, multi-fiber rack-compatible HTTP server ... Each request is executed within a lightweight fiber and can block on up-stream requests without stalling the entire server process." The gist: Falcon aims to increase throughput of web applications by using Ruby’s Fibers to be able to continue serving requests while other requests are waiting on IO (ActiveRecord queries, network requests, file read/write, etc).
The Tokyo Scout team attended Rails Tokyo #37, a Rails focused get-together that is open to any Rails topic. It was hosted at the Cookpad office in Tokyo, which has some of the best Rails engineers in Japan. In this large open area there were tables, a screen for presentations and a large kitchen island. At these Cookpad events, Cookpad provides an extensive meal prepared in house by a chef!!
In this post, I'll show how to setup Scout to monitor the performance of SQL queries, external HTTP calls, template rendering, and more in Wagtail, a Django CMS app. Wagtail is a fast, modern opensource content management system built on Django. Used at NASA, Google, MIT, and more, it's a great option for running your own CMS. When we add the scout-apm package to the app, we'll quickly gain insights on the app's performance.
I don't know of an easier way to deploy a Django app than letting Heroku do the work. That said, how do you stay on top of your app's performance, errors, and stability post-launch? Running an app on Heroku is a blissful experience, but it presents some monitoring challenges that aren't present when you control the hardware. In this post, I'll walk through a free-to-start, low-effort approach that gives you great visibility of the health of your Django app on Heroku.
Most scientific papers are unlikely to change your day-to-day approach as a Rails web developer. How not to structure your database-backed web applications: a study of performance bugs in the wild Yang et al., ICSE’18 is the exception to that rule. This study examined 12 popular, mature, opensource Rails apps for ActiveRecord performance anti-patterns. And boy, did they find some issues.
Once your Rails app begins seeing consistent traffic, you're bound to have slow SQL queries. While PostgreSQL and MySQL can log slow queries, it's difficult to gleam actionable information from this raw stream. The slow query logs lack application context: where's the LOC generating the query? Is this slow all of the time, or just some of the time? Which controller-action or background job is the caller? Enter Scout.
The Django ORM makes it easy to fetch data, but there's a downside: it's easy to write inefficient queries as the number of records in your database grows. One area where the ease of writing queries can bite is you is with N+1 queries. Expensive N+1 queries go undiscovered in small development databases. Finding expensive N+1 queries is an area where Scout is particularly helpful.
