Operations | Monitoring | ITSM | DevOps | Cloud

Apache Kafka Tutorial: Use Cases and Challenges of Logging at Scale

Enterprises often have several servers, firewalls, databases, mobile devices, API endpoints, and other infrastructure that powers their IT. Because of this, organizations must provide resources to manage logged events across the environment. Logging is a factor in detecting and blocking cyber-attacks, and organizations use log data for auditing during an investigation after an incident. Brokers, such as Apache Kafka, will ingest logging data in real-time, process, store, and route data.

Get comprehensive monitoring for your Apache Kafka ecosystem instances quickly with Grafana Cloud

We are happy to announce that the Kafka integration is available for Grafana Cloud, our composable observability platform bringing together metrics, logs, and traces with Grafana. Apache Kafka is an open source distributed event streaming platform that provides high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.

Monitoring Apache Kafka Clusters with Sumo Logic

Apache Kafka® is one of the most popular streaming and messaging platforms, commonly used in a pub-sub (publish-subscribe) model, where consumer software applications send data via messages that producer software applications can consume. Teams use Kafka for a variety of use cases, including monitoring user activity, sending notifications, and concurrently processing streams of incoming data such as financial transactions.

Distributed Tracing for Kafka Clients with OpenTelemetry and Splunk APM

This blog series is focused on observability into Kafka based applications. In the previous blogs, we discussed the key performance metrics to monitor different Kafka components in "Monitoring Kafka Performance with Splunk" and how to collect performance metrics using OpenTelemetry in "Collecting Kafka Performance Metrics with OpenTelemetry." In this blog, we'll cover how to enable distributed tracing for Kafka clients with OpenTelemetry and Splunk APM.

Collecting Kafka Performance Metrics with OpenTelemetry

In a previous blog post, "Monitoring Kafka Performance with Splunk," we discussed key performance metrics to monitor different components in Kafka. This blog is focused on how to collect and monitor Kafka performance metrics with Splunk Infrastructure Monitoring using OpenTelemetry, a vendor-neutral and open framework to export telemetry data. In this step-by-step getting-started blog, we will.