It started with a simple question: Why did one query take 10 seconds, while another almost identical query took 5? At Honeycomb, we use AWS Lambda to accelerate our query processing. It mostly works well, but it can be hard to understand and led us to wonder: What was really going on inside this box called Lambda? These questions kicked off the development of CONCURRENCY, a new aggregate in the Query Builder that lets us look at how many spans are active at once.

Two years ago, I wrote a long retrospective of observability for its third anniversary. It includes a history of instrumentation and telemetry, a detailed explanation of the technical spec, and why the whole “three pillars” thing is nonsense. At the time, it’s what was needed to steer conversations away from silly rabbit holes about data types and back to what matters: how we understand our systems.

Sampling is the practice of extracting a subset of data from a dataset to make conclusions about that larger dataset. It’s far from a perfect solution, but when it’s implemented with Refinery, Honeycomb’s trace-aware sampling proxy, sampling can help you manage very high volumes of complex event data.

Starting today, Honeycomb Metrics is now generally available to all Enterprise customers. You’ve adopted our event-based observability practices, in part to overcome the debugging roadblocks you hit when using custom metrics to identify application issues. But metrics do still provide value at the systems level. Now, you can easily see and use your metrics data alongside your event data in Honeycomb—all in one interface.

There’s no strict definition of a distributed system. But generally speaking, if you have reached a point where you’re running more than five interdependent services at once, that means you’re running a distributed system. It also means you are more than likely experiencing difficulties when troubleshooting using traditional debugging tools. Unfortunately, pulling up multiple tools, each built for a monolithic world, doesn’t help pinpoint the problem.

Register for the webinar and download the full study to see and apply Forrester’s financial model to determine the observability ROI for your organization. Many teams want to adopt observability and Honeycomb—but run into budget roadblocks because budget holders may not clearly understand the quantifiable benefits to their end users, their teams, and the bottom line.

OpenTelemetry (or “OTel”) helps you get your instrumentation started quickly, and it helps you get the most out of that telemetry data by providing flexible exporting options. As a result, it’s emerging as the new standard for instrumentation. To that end, today we’re sharing more insight into the work we’ve done (and are doing) to enable a path for all Honeycomb users toward OTel adoption. We hope you’ll be as excited as we are to embrace these open standards!

Honeycomb is known for its incredibly fast performance: you can sift through billions of rows, comparing high-cardinality data across thousands of fields, and get fast answers to your queries. And none of that is possible without our purpose-built distributed column store. This post is an introduction to what a distributed column store is, how it functions, and why a distributed column store is a fundamental requirement for achieving observability.

Your feedback is what makes Honeycomb better. We ship changes often (you can see updates in real time on our changelog), so it can be easy to miss some of the new improvements that can help you get the most out of Honeycomb. Whether it’s a big new product feature or an enhancement of existing features, you may not always be up on the latest goodness waiting for you in Honeycomb.

If there’s one thing folks working in internet services love saying, it’s: "Yeah, sure, but that won’t scale." It’s an easy complaint to make, but in this post, we’ll walk through building a service using an approach that doesn’t scale in order to learn more about the problem. (And in the process, discovering that it actually did scale much longer than one would expect.)