Operations | Monitoring | ITSM | DevOps | Cloud

AWS announced CloudTrail Lake on January 5th, 2022, as a fully-managed solution for storing and querying CloudTrail logs. At first glance, it is straightforward to set up, can be enabled for all your organization’s accounts with a radio button, and keeps data for up to seven years by default! It’s a huge time saver and headache eliminator for many, as getting CloudTrail from all organization accounts to a SIEM can be tedious and time-consuming. But all this comes with a cost.

Our mission at Cribl is to unlock the value of all your observability and telemetry data, regardless of source or destination. We aim to give you choice and control over your data—because we know data has different value to different stakeholders at different times in the data lifecycle. Users are just scratching the surface in terms of the ways they are finding value from Cribl LogStream.

Four months into this new gig at Cribl, I wish I could bottle up that “lightbulb” moment I get when walking people through how Cribl LogStream can help them gain better control of their observability data. So I hope the scenario walkthroughs below will capture some of that magic and shed some light on how LogStream can improve your organization’s data agility – helping you do more with your data, quickly, and with less engineering resources.

All Cloud providers such as AWS, Azure, Google Cloud Platform, and Oracle Cloud offer Object Storage solutions to economically store large volumes of data and retrieve it on demand. It’s far cheaper to store one petabyte of data in object storage than in block storage. As AWS S3 has become the standard, many on-premise storage appliance vendors have incorporated S3 APIs to store and retrieve data. Oracle wisely continued that trend to OCI (Oracle Cloud Infrastructure).

In the last few years, many organizations I worked with have significantly increased their cloud footprint. I’ve also seen a large percentage of newly launched companies go with cloud services almost exclusively, limiting their on-premises infrastructure to what cannot be done in the cloud — things like WiFi access points in offices or point of sale (POS) hardware for physical stores.

Health data is notoriously difficult to collect, route, and transform. I will demonstrate how to leverage the LogStream Observability Pipeline to solve these problems and help users search their Apple Health data.