Operations | Monitoring | ITSM | DevOps | Cloud

November 2023

Grafana Agent v0.38 release: new OpenTelemetry components, configuration improvements, and more

Grafana Agent v0.38 has hit the digital shelves just before the holiday season! 🧑‍🎄 The elves over at Grafana Labs have been quietly working on Grafana Agent, with more than 50 updates for all SREs and developers to use — no matter if you’re on the naughty or nice list. This includes new features, improvements, bug fixes, and significant ease-of-use changes.

How to calculate the difference of a value over time with InfluxDB and Grafana

Learning about the past helps us understand the present, and even predict the future. So, whether you are monitoring CPU usage or how long your IoT device was powered on and then off, at some point, you might want to know the difference of a value over time. InfluxDB is an open source database for storing and retrieving time series data. Thanks to its own query languages — flux and InfluxQL — it provides different and powerful ways to analyze data.

How to use flow mode for Grafana Agent with Matt Durham (Grafana Office Hours #21)

Senior Software Engineer Matt Durham shows us how to use flow mode for Grafana Agent. Flow mode is a new and better way to install and configure Grafana Agent than the older "static mode". Among other things, flow mode's modularity makes it easier to build more complex workflows like traditional data pipelines and allows for more use cases than collecting and processing telemetry.

New in Grafana roles: Manage user permissions better with 'No basic role'

Since we introduced role-based access control (RBAC) in Grafana 9.0, users — and later, service accounts — have been required to have an assigned role that includes a basic set of permissions. This sometimes led organizations to create users and service accounts that had more permissions than necessary. As a result, Grafana administrators had to make additional adjustments to users’ permissions on a case-by-case basis.

Effortlessly monitor AWS services in Grafana Cloud

Including AWS service metrics and logs into a single pane of glass helps engineers get holistic visibility into their infrastructure. Analyze 60+ AWS services across your individual accounts and regions without the toil of configuring data and building dashboards from scratch. Learn how to: Sign up for a free Grafana Cloud account today and unlock the potential of distributed tracing in your performance testing workflow.

Building scalable OSS observability with Mimir, Loki, Tempo, and Pyroscope | ObservabilityCON 2023

In this video, we cover the latest and greatest news about the scalability and performance of the open source telemetry backends that make up the Grafana LGTM Stack: Grafana Mimir for Prometheus metrics, Grafana Loki for logs, and Grafana Tempo for traces.

Do you need an OpenTelemetry Collector?

When you use OpenTelemetry SDKs to collect logs, metrics, and traces from infrastructure or an application, you’ll find many references to people using Grafana Agent and OpenTelemetry Collector. They start with an application or infrastructure that sends telemetry, and that data is sent to a collector, which then sends it to a backend like Grafana that may perform many functions, including visualization.

Application Observability and Beyla Demo | ObservabilityCON 2023

In cloud native environments, finding and resolving issues across services and between application and infrastructure dependencies can be challenging. In this recording, we provide demos on Grafana Cloud’s latest capabilities for correlating application and infrastructure observability: Application Observability and Beyla — both generally available. You will hear how Grafana unifies and contextualizes service relationships and application and infrastructure dependencies to help you resolve problems faster.

User-centered observability: load testing, real user monitoring & synthetics | ObservabilityCON 2023

Understanding your end users’ experience with your applications and services is critical, and there are a variety of tools to help. But there are also a number of different use cases: During development or in production? Simulate user behavior or monitor real user behavior? What should you use and when? This recorded session explores when and how to apply load testing, synthetic monitoring, and real user monitoring to gain insights into the end user experience of your critical applications.

Grafana SLO Demo: Prioritize critical resources with SLO-driven IRM | ObservabilityCON 2023

A majority of respondents in our Observability Survey said they were using SLOs or moving in that direction. For good reason: By highlighting the most critical error budget burndown, service level objectives (SLOs) can help you prioritize performance issues based on business impact. In this recording, Josh Abreu Mesa and Reem Tariq walk through how Grafana Cloud’s integrated SLO and Incident Response Management (IRM) capabilities can help you identify the most important issues and resolve them quickly.

How Pipedrive switched its observability stack to OpenTelemetry & LGTM | ObservabilityCON 2023

The cloud-based CRM company Pipedrive has been relentlessly modernising its observability stack, first adopting Grafana visualisation and Grafana Mimir for Prometheus metrics, then recently completed a migration of its distributed tracing from a third-party SaaS provider to OpenTelemetry and Grafana Tempo, and its logging stack from Graylog to Grafana Loki. Along the way, the team developed its own in-house library to include OpenTelemetry in its roughly 750 microservices.

Manage metrics & logging costs with Grafana Cloud + Log Volume Explorer demo | ObservabilityCON

Are your SRE and platform teams under pressure to ingest fewer metrics and logs in the name of cost savings? Reducing costs does not have to mean reduced observability. This recording walks through the cost management features in Grafana Cloud that allow you to analyze, attribute, monitor, and optimize your metrics and logs usage – and lower costs – without compromising your observability strategy.

What is CI/CD observability, and how are we paving the way for more observable pipelines?

Observability isn’t just about watching for errors or monitoring for basic health signals. Instead, it goes deeper so you can understand the “why” behind the behaviors within your system. CI/CD observability plays a key part in that. It’s about gaining an in-depth view of the entire pipeline of your continuous integration and deployment systems — looking at every code check-in, every test, every build, and every deployment.

How the LGTM Stack changed the observability culture at Wise Payments

The observability team at Wise Payments – Europe’s leader in cross-border money transfers – had long provided the company’s developers access to a multitude of tools. But as costs and complexity increased, Ibukun Itimi, Engineering Lead for Observability and Andrew Brown, Reliability Squad Lead, saw an opportunity to change not only the tools they were using, but also the observability culture.

Best practices to scale and modernize your observability strategy

ObservabilityCON 2023 took place in London this week, showcasing all the latest and greatest trends in open source observability. Following the opening keynote, the event featured a range of breakout sessions — led by both Grafana Labs experts and members of the Grafana OSS community — that explored observability best practices and lessons learned.

The Grafana OpenTelemetry Distribution for Java: Optimized for Application Observability

The OpenTelemetry project provides many different components and instrumentations that support different languages and telemetry signals. However, new users often find it hard to pick the right ones and configure them properly for their specific use cases. For this reason, OpenTelemetry defines the concept of a distribution, which is a tailored and customized version of OpenTelemetry components. Here at Grafana Labs, we are all-in on OpenTelemetry.

The Grafana OpenTelemetry Distribution for .NET: Optimized for Application Observability

The OpenTelemetry project provides many different components and instrumentations that support different languages and telemetry signals. However, new users often find it hard to pick the right ones and configure them properly for their specific use cases. For this reason, OpenTelemetry defines the concept of a distribution, which is a tailored and customized version of OpenTelemetry components. Here at Grafana Labs, we are all-in on OpenTelemetry.

What is DevOps? Grafana for Beginners Ep.2

As a beginner in DevOps, you probably have come across multiple definitions of DevOps and countless things that fall under the DevOps umbrella. So you have a basic idea of what DevOps is but are you able to explain it to another newbie like yourself? Join Lisa Jung, a senior developer advocate at Grafana, to learn about DevOps in the simplest terms possible. Subscribe to the Grafana for Beginners series to delve deeper into concepts like observability, DevOps, and how Grafana can be used to observe your system as a part of your DevOps Practice!

Set and scale service level objectives in Grafana Cloud: Introducing Grafana SLO

When we began offering Grafana Cloud Metrics, we set a service level agreement (SLA) for 99.5% of requests to be completed within a few seconds. So we built an alert that would go off if more than 0.5% of requests were slower than a couple of seconds within a five-minute moving window. Sounds reasonable, right?

ObservabilityCON 2023 - Opening Keynote (Live)

👋 Coming to you live from London, Grafana ObservabilityCON 2023's keynote introduces the latest developments in the open and composable LGTM (Loki, Grafana, Tempo, Mimir) observability stack AND many exciting announcements! Our keynote features CEO/Co-founder Raj Dutt, CTO Tom Wilkie, and members of the Grafana Labs engineering team.

Managing observability spend with Grafana Cloud's Cost Management Hub

Learn how Grafana Cloud helps analyze, manage and optimize observability spend from a central location called the cost management hub. The move to cloud-native architectures like K8s and Prometheus has caused an unprecedented increase in telemetry data that has resulted in observability bills skyrocketing. With Grafana Cloud and the central cost management hub, you will be able to answer any cost-related question with the tools to inspect, attribute, optimize and monitor your observability spend.

How to map log volume to teams with Grafana Cloud's Log Volume Explorer | Demo

Investigate the source of high log volumes in Grafana Cloud by leveraging log labels to understand which teams or applications are responsible for log usage. In this video, see how to use the Log Volume Explorer with a point-and-click user interface and explore log volumes by using any combination of labels associated with the logging data. Slice and dice the data as you choose to see log volume broken down by teams, applications, clusters, cloud region.

Manage log volumes, metrics cardinality, monthly bills: Explore Grafana Cloud cost management tools

As more organizations adopt observability at massive scale, they have also been grappling with rising costs. Over the past 12 months, we have been working on different solutions to help our users better understand and manage their observability stack, not to mention the bills that come with scaling it.

Grafana Beyla 1.0 release: zero-code instrumentation for application telemetry using eBPF

Just two months after introducing the public preview of Grafana Beyla, we are excited to announce the general availability of the open source project with the release of Grafana Beyla 1.0 at ObservabilityCON 2023 today. We’ve worked hard in the last two months to stabilize, stress test, and refine the features that were part of the public preview of this open source eBPF auto-instrumentation tool.

How Asserts.ai will make it even easier for Grafana Cloud users to understand their observability data

At Grafana Labs, our mission has always been to help our users and customers understand the behavior of their applications and services. Over the past two years, the biggest needs we’ve heard from our customers have been to make it easier to understand their observability data, to extend observability into the application layer, and to get deeper, contextualized analytics.

Announcing Application Observability in Grafana Cloud, with native support for OpenTelemetry and Prometheus

The Grafana LGTM Stack (Loki for logs, Grafana for visualization, Tempo for traces, and Mimir for metrics) offers the freedom and flexibility for monitoring application performance. But we’ve also heard from many of our users and customers that you need a solution that makes it easier and faster to get started with application monitoring.

Grafana k6 for Beginners: Why observability needs testing

Having observability and monitoring solutions is a great way to gain insights into your applications' health, behavior, and performance. However, it doesn’t prevent incidents. Observability needs a partner, and this is where Grafana k6 can help you! In this video, Marie Cruz, a Developer Advocate at Grafana Labs, explores what Grafana k6 is, why it's the missing puzzle piece in your Grafana stack, and how to get started.

Load testing on Kubernetes with k6 Private Load Zones (Grafana Office Hours #19)

This week, we're talking about how you can do load testing on Kubernetes with k6 Private Load Zones, a new feature on Grafana Cloud k6 that leverages the k6 Kubernetes operator to allow you to run distributed load tests against applications behind a firewall. Here to discuss this new feature are Senior Software Engineer Olha Yevtushenko, Product Manager Daniel González Lopes, Developer Advocate Paul Balogh, and Senior Developer Advocate Nicole van der Hoeven.

How Grafana Labs switched to Karpenter to reduce costs and complexities in Amazon EKS

At Grafana Labs we meet our users where they are. We run our services in every major cloud provider, so they can have what they need, where they need it. But of course, different providers offer different services — and different challenges. When we first landed on AWS in 2022 and began using Amazon Elastic Kubernetes Service (Amazon EKS), we went with Cluster Autoscaler (CA) as our autoscaling tool of choice.

Resolve issues faster with Grafana Cloud Application Observability

Grafana Cloud Application Observability provides an out-of-the box experience to monitor application performance and minimize MTTR. With its native support of the open standards OpenTelemetry and Prometheus, Application Observability unifies signals across the full stack, accelerating root cause analysis while removing proprietary formats and vendor lock-in. Watch this demo of how to use Application Observability in Grafana Cloud.

Zero-code application observability with Grafana Beyla and eBPF: demo

The eBPF-based OSS auto-instrumentation tool Grafana Beyla makes it easier to get started with application observability. Beyla provides RED (Rate, Errors, Duration) metrics through OpenTelemetry or Prometheus for your existing web services, whichever language they are written in. You don’t need to change any line of application code or configuration; you only need to deploy the Beyla in the same host as the service that you want to monitor. Collecting monitoring data with the eBPF autoinstrument tool has very low overhead, and allows you to capture data about your runtime, which is impossible with manual code instrumentation. Watch this in-depth demo of how to use Grafana Beyla to get started with application observability.

Control Prometheus cardinality and metrics cost with Adaptive Metrics

Adaptive Metrics is a cost management feature in Grafana Cloud that helps enterprises control Prometheus cardinality and reduce their observability spend by identifying and eliminating unused metrics. Grafana Cloud customers using Adaptive Metrics see 20-50% reduction in their observability bill.

Grafana panel titles: Why we changed from center to left-aligned

As Grafana evolved over the years, so did our panel headers. In our quest for improvement, we continually added design options that created more comprehensive panels, but also an increasingly complex interface. It was a process of continual adaptation without a roadmap — which, though well-intentioned, began to result in unforeseen challenges.

Saga Design System: shaping the future of user experiences at Grafana Labs

At Grafana Labs, we want to empower our fellow Grafanistas and the community to get the most out of the Grafana LGTM Stack (Loki for logs, Grafana for visualization, Tempo for traces, and Mimir for metrics). As part of this effort, we recently launched a new Grafana developer portal. And now, we’re pleased to announce the launch of the Saga Design System, which establishes a shared visual language for all of Grafana Labs’ offerings.

How we upgraded to MySQL 8 in Grafana Cloud

Starting around June this year, we upgraded our Grafana databases in Grafana Cloud from MySQL 5.7 to MySQL 8, due to MySQL 5.7 reaching end-of-life in October. This project involved tens of thousands of customer databases across dozens of MySQL database servers, multiple cloud providers, and many Kubernetes clusters.

Combining frontend and backend performance with John Hill (Grafana Office Hours #18)

In this episode of Grafana Office Hours, Developer Advocates Marie Cruz and Nicole van der Hoeven speak with John Hill, a Web UI Test Engineer and Grafana k6 champion, to talk about how the Grafana and k6 ecosystems can be used to ensure performance in mission-critical applications like NASA’s Open MCT.

Monitor your OpenAI usage with Grafana Cloud

In the ever-changing field of artificial intelligence, OpenAI is consistently seen as a leader in innovation. Its AI models, starting with GPT-3 and now with GPT-4, are already used extensively in software development and content creation, and they’re expected to usher in entire sets of new systems in the future.

What is Observability? Grafana for Beginners Ep. 1

When you are getting started with observability, the jargon and concepts used to explain observability may go straight over your head. Let’s take out the complexity and talk about observability in the simplest terms possible. Join Lisa Jung, a senior developer advocate at Grafana, to get your learning on with the Grafana for Beginners series. You will learn about concepts such as observability and DevOps and how Grafana can be used to observe your system as a part of your DevOps practice.

Grafana Tempo 2.3 release: faster trace queries, TraceQL upgrades

Grafana Tempo 2.3 has been unleashed upon the world, bringing with it the latest iteration of the vParquet backend! Tempo 2.3 has a little bit of everything, but the headline item here is vParquet3 and new features that improve search speeds. Watch the video above for all the details, or continue reading to get a quick overview of the latest updates in Tempo. If you’re looking for something more in-depth, don’t hesitate to jump into the changelog or our Grafana Tempo 2.3 release notes.