Kafka deployments often rely on additional software packages not included in the Kafka codebase itself—in particular, Apache ZooKeeper. A comprehensive monitoring implementation includes all the layers of your deployment so you have visibility into your Kafka cluster and your ZooKeeper ensemble, as well as your producer and consumer applications and the hosts that run them all.
Jenkins is an open source, Java-based continuous integration server that helps organizations build, test, and deploy projects automatically. Jenkins is widely used, having been adopted by organizations like GitHub, Etsy, LinkedIn, and Datadog. You can set up Jenkins to test and deploy your software projects every time you commit changes, to trigger new builds upon successful completion of other builds, and to run jobs on a regular schedule.
When monitoring highly distributed applications, which might rely on hundreds of services and infrastructure components across multiple cloud-based and on-premise environments, identifying problems and pinpointing the origin of an issue can be challenging. Even if you already have robust monitoring and alerts, your infrastructure and applications will likely change over time, which may make it difficult to reliably detect irregular behavior.
Apache Flink is an open source framework, written in Java and Scala, for stateful processing of real-time and batch data streams. Flink offers robust libraries and layered APIs for building scalable, event-driven applications for data analytics, data processing, and more. You can run Flink as a standalone cluster or use infrastructure management technologies such as Mesos and Kubernetes.
VMware vSphere is a server virtualization platform that enables organizations to provision and manage virtual machines at scale. With its comprehensive suite of products, vSphere helps companies manage datacenter resources, migrate workloads without downtime, run applications with high availability, and more. To keep tabs on dynamic vSphere environments and effectively address resource bottlenecks, you need deep visibility across every part of your infrastructure.
Scylla is an open source database alternative to Apache Cassandra, built to deliver significantly higher throughput, single-digit millisecond latency, and always-on availability for real-time applications. Unlike Cassandra which is written in Java, Scylla is implemented in C++ to provide greater control over low-level operations and eliminate latency issues related to garbage collection.
Gunicorn is a popular application server for Python applications. It uses the Web Server Gateway Interface (WSGI), which defines how a web server communicates with and makes requests to a Python application. In production, Gunicorn is often deployed behind an NGINX web server. NGINX proxies web requests and passes them on to Gunicorn worker processes that execute the application.
In the previous post in this series, we dug into the data you should track so you can properly monitor your Kubernetes cluster. Next, you will learn how you can start inspecting your Kubernetes metrics and logs using free, open source tools. In this post we’ll cover several ways of retrieving and viewing observability data from your Kubernetes cluster.
If you’ve read Part 3 of this series, you’ve learned how you can use different Kubernetes commands and add-ons to spot-check the health and resource usage of Kubernetes cluster objects. In this post we’ll show you how you can get more comprehensive visibility into your cluster by collecting all your telemetry data in one place and tracking it over time.
Tags provide critical context for troubleshooting issues across any dimension of your environment. By applying best practices for tagging your systems, you can efficiently organize and analyze all your monitoring data, and set up automated multi alerts to streamline alerting workflows. Similar to any tags you would add to your services and infrastructure, monitor tags—tags that you apply to your monitors—are an essential feature for organizing and simplifying your workflows.
