The latest News and Information on Observabilty for complex systems and related technologies.
We’ve all grown used to logs, metrics and traces serving as the “three pillars of observability.” And indeed they are very important telemetry signals. But are they indeed the sum of the observability game? Not at all. In fact, one of the key trends in observability is moving beyond the ‘three pillars: One emerging telemetry type shows a particularly interesting potential for observability: Continuous Profiling.
It is only possible to come to an understanding of a system of interest by trying to change it. Here, Jackson contrasts action research with old-style hard science, which tries to study a system from the outside. Laboratories draw a line between experiment and scientist. In the social world, there is no outside: we participate in the systems we study. I’ve noticed this in code: when I come to an existing codebase, I get a handle on it by changing stuff.
When I speak at conferences, I often fall back to the fact that just a couple of decades ago we’d observe production by kicking the server. This is obviously no longer practical. We can’t see our production. It’s an amorphous cloud that we can’t touch or feel. A power that we read about but don’t fully grasp. In this case, we have physical evidence that the cloud is there. A part of this major shift in our industry is a change to our fundamental roles as engineers.
Sometimes the simplest questions prompt the most spirited discussion. Questions like: What is the airspeed velocity of an unladen swallow? What should we have for dinner tonight? Or, as we find out in this episode of “Grafana’s Big Tent" what even is observability?
Monitoring is often not the first thing on the mind of the modern developer. Yet, it’s necessary at many points of the software development lifecycle, including: before deprecating an API, before launching a new feature, after launching the feature, and more. In fact, monitoring needs can vary much more than the classic Ops monitoring.
Are you overspending on monitoring and APM tools? Forrester’s Total Economic Impact analysis of Honeycomb identified significant ROI in customers using us to reduce spend on less efficient APM workflows. But this isn’t about budget reallocation to a newly branded set of similar but shinier tools.
Remember the old days where if you had an uptime of 99.9 you could be fairly confident everyone was having a good experience with your application? That’s not really how it works anymore. Modern, distributed systems are so complex they typically fail unpredictably, making it much harder to diagnose issues. Traditional monitoring grew out of those early days, allowing you to check the health of simpler systems.
Honeycomb uses AWS Lambda as a core part of our query execution architecture; Lambda’s ability to quickly allocate lots of resources and charge us only for use is invaluable to keeping Honeycomb fast and affordable. Our total Lambda bill is easily accessible in the AWS Console, but how do we know which customers or application areas dominate this bill? How do we judge the cost of changes we make to our own software?
