Operations | Monitoring | ITSM | DevOps | Cloud

Five years ago today, Grafana Loki was introduced to the world on the KubeconNA 2018 stage when David Kaltschmidt, now a Senior Director of Engineering at Grafana Labs, clicked the button to make the Loki repo public live in front of the sold-out crowd. At the time, Loki was a prototype: We bolted together Grafana as a UI, Cortex internals, and Prometheus labels to find out if there was a need for a new open source tool to manage logs.

Table of contents This is the second part of our 12-day Advent of Monitoring series. In this series, Checkly's engineers will share practical monitoring tips from their own experience. We encountered a tricky issue with our public dashboards: they were experiencing sporadic outages, happening about once every two days. The infrequency and unpredictability of these outages made them particularly challenging to diagnose.

Working in the cloud is certainly convenient, but the convenience comes at a price. With more and more organizations transitioning to the cloud, and a rise in preference towards cloud-native applications, hosting most, if not all the components of your business in the cloud is becoming increasingly common.

I recently delved into the idea of using labels within Prometheus to craft objects and hierarchies where none initially existed. Check out that piece here. The essence was harnessing the prowess of OTEL to achieve more, faster. The ambition? Transform these abstract virtual objects and integrate them into SquaredUp's knowledge graph, thereby unlocking the potential of data mesh and correlation.

With the release of Grafana 10.2, we made a number of enhancements to Grafana Alerting. These updates included the rollout of Insights, a new section of the Grafana Alerting home page. Available now to all Grafana Cloud users, Insights offers valuable information, such as statistics on alert rules and notifications, to help you monitor alerting data and quickly analyze alert performance.

In the realm of data and complex scenarios, we humans naturally gravitate towards visualizing things as entities with attributes, rather than just raw data. Consider the phrase, “The response time on our Ad Generation service has increased.” It immediately resonates with the audience supporting the service.

In observability, finding the root cause of a problem is sometimes likened to finding a needle in a haystack. Considering that the problem might be visible in only a tiny fraction of millions or billions of individual traces, the task of reviewing enough traces to find the right one is daunting and often ends in failure.