OpenTelemetry enables Observability, and building observable systems requires you to understand the various ways in which they can fail. Jumping from one possible fix to another and one change to another without fully recognizing the impact on the system can be a significant hindrance to a successful customer experience. In this post, I’ll explain how to get started with OpenTelemetry to help you make your systems more observable.

I bet we all have heard so much about eBPF in recent years. Data shows that eBPF is quickly becoming the first choice for implementing tracing and security applications, and Elastic is also working relentlessly on supercharging our security solutions (and more) with eBPF. However, one major challenge is that the eBPF ecosystem lacks tooling to make developers' lives easier. eBPF programs are written in C but compiled for a specific ISA later executed by the eBPF Virtual Machine.

Before we talk about OpenTelemetry, we should talk about telemetry. Telemetry is: And an instrument is: For the purpose of measuring running computer software and systems, our instruments are virtual instruments. That is to say, code that measures other code. It sounds simple: read a measurement and send it to a remote location. In practice, to make that telemetry data useful in today’s cloud-native and ever more complex environments, there are huge logistical and technical hurdles to overcome.

OpenTelemetry is enabling a revolution in how Observability data is collected and transmitted. See our What Is OpenTelemetry post on why this is an important inflection point in the Observability space. In this post, we’ll walk through how to configure the OpenTelemetry Gems within a Rails app.

Grafana Tempo 1.3 has been released! We are proud to add the capability to search the backend datastore. This feature will also appear soon in Grafana Cloud Traces. If you want to dig through the nitty-gritty details, you can always check out the v1.3 changelog. If that’s too much, this post will cover the big ticket items. You can also register for our upcoming webinar “Distributed tracing in Grafana: From Tempo OSS to Enterprise” on Jan.

In distributed tracing, sampling is frequently used to reduce the number of traces that are collected and stored in the backend. This is often desirable because it is easy to produce more data than can be efficiently stored and queried. Sampling allows us to store only a subset of the total traces produced.

TL;DR: Use auto-instrumentation from OpenTelemetry. Traces will happen. Then your code can use global library functions to customize those traces with your specific important data.

Anders Østhus is a DevOps Engineer on the Digital Tools team at Proactima AS, a consulting firm based in Norway that offers services and expertise in risk management, cybersecurity, healthcare, environmental solutions, and more. It can be difficult to orient yourself in the distributed tracing space, and getting all the parts of a tracing setup to play well with each other can be a bit tricky. But the benefits of tracing are undeniable.