Searching Sentry Logs with Regex

We use Sentry's new Regex search for Logs to hunt for non-obvious bugs in our app.

https://docs.sentry.io/concepts/search/#regular-expressions-logs-only

What do you do when traces show that Postgres jobs are backing up? Logs are trace-connected, so by looking at one of the affected traces, we can see all of the logs on that trace. In there, we can see Postgres has logged that it acquired a lock only after a long wait.

Using regex, we can find all of these logs that describe acquiring a lock after a 10-second-plus wait. That narrows our search from thousands or hundreds of logs down to about a dozen.

Looking closer at the logs on at of those traces, we can see that the psql process is what backed up all our jobs, and if we look at the command ran, it appears to be a manual migration. That tells us that someone manually, using the CLI, ran a migration.

Try Sentry for free: https://sentry.io/welcome
Docs: https://docs.sentry.io