Operations | Monitoring | ITSM | DevOps | Cloud

You can’t throw a rock without hitting an online discussion about ‘vibe coding,’ so I figured I’d add some signal to the noise and discuss how I’ve been using AI-driven coding tools with observability platforms like Honeycomb over the past six months. This isn’t an exhaustive guide, and not everything I say is going to be useful to everyone—but hopefully it will clear up some common misconceptions and help folks out.

At Honeycomb, we believe that observability should be accessible, effective, and transformative. That’s why our customer education team is thrilled to introduce Honeycomb Academy, a learning hub designed to help our customers of all experience levels master observability, OpenTelemetry, and the Honeycomb platform.

At Honeycomb, we’re actively growing our design system, Lattice, to ensure accessibility, optimize performance, and establish consistent design patterns across our product. One metric we use to measure Lattice is the adoption of components across the product. Adoption is about understanding how, where, and why they’re being used.

CloudWatch metrics can be a very useful source of information for a number of AWS services that don’t produce telemetry as well as instrumented code. There are also a number of useful metrics for non-web-request based functions, like metrics on concurrent database requests. We use them at Honeycomb to get statistics on load balancers and RDS instances. The Amazon Data Firehose is able to export directly to Honeycomb as well, which makes getting the data into Honeycomb straightforward.

Take a drink if you hate that term. Tim O’Reilly has written a post about the huge shift AI is making with our relationship to software engineering (read: The End of Programming as we Know It). Scary title. But the content is less so. Why?

Last week marked the inaugural HumanX conference, a convening of leaders, technologists, policy makers, and media, all brought together to discuss the state of AI and its potential impact on the future of software, business, and society.

Observability and monitoring are not about gathering different data—they differ in their purpose, but share the same data. Monitoring is focused on notification based on predefined questions. Whether that’s through Dashboards people watch, or push-based alerts to notification systems like SMS or purpose-built platforms like PagerDuty.

I’ve increasingly changed my perspective on LLMs and modern AI systems over the past few years: Let me elaborate on why I believe this now.

If you’ve been using Honeycomb for a bit, you know that Calculated Fields (otherwise known as derived columns) are a powerful way to transform your events to a format that’s easier to query and understand. However, they use a lisp-esque language that can be difficult to read and a pain to write. If you dislike making Calculated Fields and want something a little easier, here’s a generative AI prompt that can generate them from natural language.

As software teams race to integrate AI into their development workflows, we need to ask ourselves: are AI-powered tools actually making software better? The latest research from DORA confirms what many engineers have long suspected, and what we at Honeycomb have said for a long time: AI tools don’t magically lead to better software. In fact, without careful implementation, AI can introduce a whole slew of challenges, including decreased productivity and unreliable code.