Today we’re excited to introduce the Kubernetes integration for Grafana Cloud, our composable observability platform bringing together metrics, logs, and traces with Grafana. Grafana Cloud users can now easily monitor and alert on core Kubernetes cluster metrics using the Grafana Agent, our lightweight observability data collector optimized for sending metric, log, and trace data to Grafana Cloud.
Greetings! This is Abdelkrim from the Solutions Engineering team, and I am with Sriram from the Enterprise Plugin team. We both joined Grafana Labs in February this year, and we already have some stories to share with you. I came to Grafana Labs from a big data and analytics background, and I witnessed a lot of companies storing monitoring and performance data in all kinds of analytics platforms (data lakes, data warehouses, cloud, etc.).
Grafana Explore makes correlating metrics and logs easy. Prometheus queries are automatically transformed into Loki queries . And we will be extending this feature in Grafana 8.0 to support smooth logs correlation not only from Prometheus, but also from Graphite metrics. Prometheus and Loki have almost the same query syntax, so transforming between them is very natural. However, Graphite syntax for queries is different, and in order to map it to Loki, some extra setup is required.
For many years, it has been possible to scale Cortex clusters to hundreds of replicas. The relatively simple Dynamo-style replication relies on quorum consistency for reads and writes. But as such, more than a single replica failure can lead to an outage for all tenants. Shuffle sharding solves that issue by automatically picking a random “replica set” for each tenant, allowing you to isolate tenants and reduce the chance of an outage.
Whether it’s during an incident to find the root cause of the problem or during development to troubleshoot what your code is doing, at some point you’ll have an issue that requires you to search for the proverbial needle in your haystack of logs. Loki’s main use case is to search logs within your system. The best way to do this is to use LogQL’s line filters. However, most operators are case sensitive.
Daniel is a Site Reliability Engineer at k6.io. He’s especially interested in observability, distributed systems, and open source. During his free time, he helps maintain Grafana Tempo, an easy-to-use, high-scale distributed tracing backend. Distributed tracing is a way to track the path of requests through the application. It’s especially useful when you’re working on a microservice architecture.
We built Grafana Enterprise Metrics (GEM) to empower centralized observability teams to provide a multi-tenanted, horizontally scalable Prometheus-as-a-Service experience for their end users. The GEM plugin for Grafana is a key piece of realizing this vision. It provides a point-and-click way for teams operating GEM to understand the state of their cluster and manage settings for each of the tenants within it.
I have been using Grafana for almost four years now, and in that time it has become my go-to tool for my application observability needs. Especially now that Grafana allows you to also view logs and traces, you can easily have all three pillars of observability surfaced through Grafana. As a result, when I started working on the Elixir PromEx library, having Grafana be the end target for the metrics dashboards made perfect sense.
Since we launched Grafana Enterprise Metrics (GEM), our self-hosted Prometheus service, last year, we’ve seen customers run it at great scale. We have clusters with more than 100 million metrics, and GEM’s new scalable compactor can handle an estimated 650 million active series. Still, we wanted to run performance tests that would more definitively show GEM’s horizontal scalability and allow us to get more accurate TCO estimates.
