Operations | Monitoring | ITSM | DevOps | Cloud

You've deployed a new feature into production. You've done your unit testing, fixed lots of bugs, your code is awesome. Now it's time for hundreds/thousands/millions of users to break...err...use your feature. You're diligent about tracking usage in real-time, and getting customer feedback when something goes wrong. You track the performance and response time impacts on the server. All is good...except...that feature isn't quite working for a specific group of users. Now what?

At Mezmo, we handle an enormous volume of telemetry data for our customers and ourselves, requiring a robust and efficient search and analytics backend. For years, ElasticSearch served us well, but as our infrastructure grew to a multi-cluster, multi-petabyte scale, we started to see the cracks—rising costs, performance bottlenecks, and scalability concerns. We needed a change, one that would make our system more cost-effective while maintaining speed and reliability.

This is an updated version of an earlier blog post to reflect current definitions of a telemetry pipeline and additional capabilities available in Mezmo Our recent blog post about observability pipelines highlighted how they centralize and enable telemetry data actionability. A key benefit of telemetry pipelines is users don't have to compare data sets manually or rely on batch processing to derive insights, which can be done directly while the data is in motion.

There was a lot happening in the world in 2024, but today, I’m going to highlight some product accomplishments we had this year at Mezmo on our Telemetry Pipeline product.

I attended Amazon Web Services re:Invent conference. This is AWS's annual user conference, which takes over most of Las Vegas for a week. There’s a lot to do and take in—customer stories galore, new tech, learning different use cases, and all the walking. But you’re here to hear what I learned, so I’ve broken it down into sections. Enjoy!

Last week, I attended one of the last conferences of the year with team Mezmo: the Gartner IT Infrastructure, Operations & Cloud Strategies Conference in Las Vegas. Not surprisingly, there were over 20 sessions covering observability and how it is getting increasingly critical in the new complex distributed computing environment. Of course, there were many sessions, including all keynotes that addressed the advent and impact of AI on IT operations and observability.

A few weeks ago, members of Mezmo were at Kubecon and attended several sessions. You can see a post with my recap and session highlights. Today, though, I’m going to discuss three sessions that my colleagues found interesting for our peers in Observability.

In April, we published a Part I blog on the topic of how we planned to use our own Telemetry Pipelines product here at Mezmo to manage metrics data. Remember that utopia of observability we dreamed about? Well, we’re not quite there yet, but we’ve made progress.

I had a fantastic opportunity to sit with Ben Good of Google and Rich Prillinger of Mezmo and participate in the discussion about the new DORA 2024 report. The 10th edition of the DORA report covers the impact of AI on software development, explores platform engineering’s promises and challenges, and emphasizes developer experience and stable priorities for success.

Every day, telemetry data is emitted, processed, and then either stored or deleted from services. Unfortunately, understanding and then optimizing the telemetry data can be difficult. These difficulties lead to higher MTTR/D, burnout, and unwanted overage bills.