Kafka Migration and Lessons Learned

Over the last few months, Honeycomb’s platform team migrated to a new iteration of our ingest pipeline for customer events. Our migration to this newer architecture did not go too smoothly, as can be attested by our status page since February. There were also many near-incidents where we got paged and reacted quickly enough to avoid major issues. We’ve decided to write a full overview of all the challenges we had encountered, which you can can download.


One Year of Graviton2 at Honeycomb

A year ago, we wrote about our experiences as early adopters of Graviton2, and how we were able to see 30% price-performance improvements on one dogfood workload from switching to the arm64 architecture. In those initial experiments, we validated running 20% fewer shepherd ingest workers, using the m6g instance type, which cost 10% less per instance compared to c5 instances.


Uniting Tracing and Logs With OpenTelemetry Span Events

The current landscape of what our customers are dealing with in monitoring and observability can be a bit of a mess. For one thing, there are varying expectations and implementations when it comes to observability data. For another, most customers have to lean on a hodgepodge of tools that might blend open source and proprietary, require extensive onboarding as team members have to learn which tools are used for what, and have a steep learning curve in general.


Event Latency: What It Is and Why You Should Care

Recently, we added a new derived column function to Honeycomb, INGEST_TIMESTAMP(), which can help customers debug event latency and/or inaccurate timestamps. A meaningful minority of the events sent to Honeycomb are already old when they arrive, and a very special few claim to have been sent from the future. Has this happened to you? Let’s do an experiment.

Cloud Native Goes Native with Charity Majors and David McKay

Cloud-native and serverless technologies are gaining traction as organizations increasingly recognize the value of containers and Kubernetes in application development environments. As a result, the cloud-native ecosystem is growing at a healthy pace. In this topic spotlight, we take a look at the cloud-native landscape and discuss its impact on DevOps, application security and more. Some of the issues discussed during the webinar include.

Refine Your Observability Experience at Scale

Today, we announced that Refinery is now generally available. With Refinery, it’s now easy to highlight the critical debugging data you need and to stop paying for the rest. Refinery is a sampling solution that lets you control resource costs at scale without sacrificing data fidelity. Support for Refinery is now also included in Honeycomb Enterprise plans.


Sweetening Your Honey

Are you looking for a better way to troubleshoot, debug, and really see and understand what weird behavior is happening in production? Service-level objectives (SLOs) and observability can help you do all that—but they require collecting and storing the right data. If we’re naive with our telemetry strategy, we spend a lot of money on storing data without seeing adequate return on investment in the form of insights.


Show Your Query You Love It By Naming It

Honeycomb is all about collaboration: We believe that observability is a team sport, and we want to give you as many tools to help your team get the ball down the field (i.e., untangle knotty problems) as we can. We want you to be able to share the current state of your work so that others can follow and figure out what’s up, and we want you to leave breadcrumbs so the next time you’re stuck here, you can find your way back.


Getting Started with Java & OpenTelemetry

It’s easy to get started with Java and Honeycomb using OpenTelemetry. With Honeycomb being a big supporter of the OpenTelemetry initiative, all it takes is a few parameters to get your data in. In this post, I will walk through setting up a demo app with the OpenTelemetry Java agent and show how I was able to get rich details with little work by combining automatic instrumentation from the agent with custom instrumentation in the code.