Operations | Monitoring | ITSM | DevOps | Cloud

July 2022

"Why Are My Tests So Slow?" A List of Likely Suspects, Anti-Patterns, and Unresolved Personal Trauma

“Lead time to deploy” means the interval from when the code gets written to when it’s been deployed to production. It has also been described as “how long it takes you to run CI/CD.” How important is it? It’s nigh-on impossible to have a high-performing team if you have a long lead time, and shortening your lead time makes your team perform better, both directly and indirectly.

Ask Miss O11y: My Manager Won't Let Me Spend Any Time Instrumenting My Code

My organization doesn’t want me spending time on instrumenting my product. What can I do? Thanks for the question! You’ll be relieved to hear that you’re in the majority, and also that there are quick (and easy) steps you can do to prove that instrumenting your code is worthwhile.

Building a Resilient System: Our Journey to Observability at Intercom

At Intercom, we focus on customer experience above all—our service’s availability and performance is our top priority. That requires a strong culture of observability across our teams and systems. As a result, we invest a lot in the reliability of our application. But unpredictable failures are inevitable, and when they happen it’s humans that fix them. We operate a socio-technical system, and its ability to recover when faced with adversity is called resilience.

Authors' Cut-Debugging with the Core Analysis Loop, and What to Build vs Buy

In the old days, the most senior members of an engineering team were the best debuggers. They had built up such an extensive knowledge about their systems that they instinctively knew the right questions to ask and the right places to look. They even wrote detailed runbooks in an attempt to identify and solve every possible issue and possible permutation of an issue.

Datasets, Traces, and Spans-Oh My!

If you've stumbled (or purposefully landed) on this blog post, chances are you are new to—or diving deeper—into the observability space, o11y for short. Suffice it to say, you’re not in Kansas anymore. Honeycomb in a lot of ways can serve as a yellow brick road into o11y, and this article should serve as an introduction into how Honeycomb facilitates implementing o11y into applications and distributed services.

A to Z With Observability and OpenTelemetry

How do you go from A to Z with observability and OpenTelemetry? This post answers a question we hear often: “How do I get started on instrumentation with OpenTelemetry, while also following best practices for the long-term?” This article is all about taking you from A to Z on instrumentation. This will help you: We will use a simple greeting service application written in Node.js to understand the journey. You can find the pre-instrumented state here.

Top Takeaways from Monitorama 2022

Since 2013, Monitorama has been a community-driven conference, bringing together open source development and operations engineers to focus on pushing the boundaries of monitoring software and practices. It’s chock full of thought-provoking content in the conference talks. The casual atmosphere also makes the hallway track a great way to network with fellow engineers and vendors alike to pick up on new developments in the monitoring space.

On Counting Alerts

A while ago, I wrote about how we track on-call health, and I heard from various people about how “expecting to be woken up” can be extremely unhealthy, or how tracking the number of disruptions would actually be useful. I took that feedback to heart and wanted to address the issues they raised, and also provide some numbers that explain the position I took with these metrics on alerts.

We Learn Systems by Changing Them

It is only possible to come to an understanding of a system of interest by trying to change it. Here, Jackson contrasts action research with old-style hard science, which tries to study a system from the outside. Laboratories draw a line between experiment and scientist. In the social world, there is no outside: we participate in the systems we study. I’ve noticed this in code: when I come to an existing codebase, I get a handle on it by changing stuff.