Operations | Monitoring | ITSM | DevOps | Cloud

I'm investigating repeated errors in my e-commerce application, and I need to get enough context in a single Honeycomb query to piece the entire picture together. Each query returns events based on the event's WHERE clauses, but I want to know several things from outside of the event that recorded an error. Things like: Those attributes are all over the trace. That's going to make a single query tough, right? Wrong!

A critical system has to stay reliable without new failures or added downtime, and live debugging, confirming the root cause without stopping the system, is often the only way to do that. In practice, this means having runtime context: on-demand evidence generated at the point of failure rather than logging configured months earlier, which is what keeps MTBF up, MTTR, and MTTA down.

It's been one year since we issued an AI mandate inside Honeycomb, and we've been doing a lot of reflecting internally on how far we've come and how far we have yet to go.

If you are able to build it better than your vendor, then change your vendor. Not build it. Rishi builds large-scale observability systems at Last9, focusing on reliable and cost-efficient telemetry infrastructure, and writes about the practical lessons learned while operating ClickHouse, VictoriaMetrics, and OpenTelemetry in production.

Sampling is a core skill that everyone who runs an observability pipeline at scale will learn. There are lots of tradeoffs within the various decisions you'll make from reducing bandwidth, CPU, and memory, to reducing costs and making the observability backend's performance better for users. Historically, there have only been three mechanisms, each with their own tradeoffs: However, there is a secret fourth option: adaptive tail sampling—which changes those tradeoffs.

A few months ago, Darragh Curran, CTO at Fin (formerly Intercom) set a public goal to double engineering productivity and nearly tripled it instead. They did so by pulling a few levers: AI writing code at scale, building an AI-driven PR review system, leveraging observability as a trust mechanism, and with leadership becoming more hands-on through the transition.

NestJS 12 dropped last week with ESM packages, Standard Schema validation, a rebuilt CLI, and a brand-new observability SDK. That last one is the most interesting part for anyone running NestJS in production with an APM tool. At Scout Monitoring, we instrument NestJS applications through our Node.js agent. A framework shipping its own observability layer changes the conversation for every APM vendor in the ecosystem, including us. Here is what you need to know.