As a solutions engineer, I always handle the complex challenge of collecting IT and security data. The variety of modern ephemeral systems increases the complexity of collection requirements. Cloud, PCF, and Kubernetes emit metrics, logs, and traces through methodologies like Cloud Foundry’s Nozzle, Prometheus scrapers, and OpenTelemetry collectors. I often find all of these deployed in parallel in a single enterprise environment to meet the evolving needs of IT Ops or SecOps.

As a product manager for Cribl Stream, I enjoy listening to customer challenges and bringing new features and enhancements into the product to help reduce pain points and better enable customers.

Cribl Search empowers users to explore and analyze data directly at its source. However, finding sample data for testing these queries can be time-consuming. To overcome this, Cribl Search provides $vt_dummy, a built-in virtual table designed to generate dummy events on demand.

One of our most commonly asked questions is when we will support Worker Groups in Azure. We’ve heard you loud and clear; some exciting news will make your data management much more straightforward. We’re introducing a Cribl-managed Cribl Stream data plane, also known as Worker Groups, in Microsoft Azure. These Worker Groups are oil to your engine—essential for data operations, handling everything from shaping and transforming to enriching and processing your data.

Cribl Lake is simple, it’s storage, it’s simplified storage to keep large volumes of IT and security data for long retention periods. And now it’s even easier for you to start using Cribl Lake. In addition to Cribl-managed Cloud Worker Groups, cloud customers can now use self-managed Hybrid Worker Groups to send data directly to Cribl Lake. This means all your worker groups, whether hybrid or cloud, can write data to Cribl Lake — all coordinated by your Cribl.Cloud Leader.

Think of it as your digital concierge to achieve faster time-to-value IT and security teams face more challenges than ever, with data growing at 28% CAGR and taking numerous shapes and forms. Cribl’s suite of products – Stream, Edge, Search, and Lake – is built on a unified data processing engine specifically designed for IT and security data.

The demands on IT, observability, and security teams have never been greater. With data volumes exploding at a 28% CAGR and hybrid environments becoming the norm, organizations are facing significant challenges: those rapidly growing data volumes I mentioned, the intricacies of hybrid and cloud-native architectures, and the need for real-time insights. Oh, and don’t forget the constant threat of security breaches.

This is the first of a planned series of blog posts that explain how the Cribl SRE team builds, optimizes, and operates a robust Observability suite using Cribl’s products, Cribl.Cloud operates on a single-tenant architecture, providing each customer with dedicated AWS accounts furnished with ready-to-use Cribl products. This provides our customers with strict data and workload isolation but presents some interesting and unique challenges for our Infrastructure and operations.

The greatest innovations are often the simplest. They address fundamental needs and make life easier in the most direct way. Cribl Lake was just announced as the winner of CRN’s 2024 Tech Innovators Award for Data Information Management. We are so happy and honored by this recognition, which solidifies our belief that the best innovations are indeed the simplest.