One of the most captivating discussions I had at KubeCon Europe 2023 in Amsterdam was about standardization of a query language for observability. This query language standard aims to provide a unified way of querying observability data across logs, metrics, traces, and other relevant signals. The conversation shed light on the pressing need for a standardized approach to overcome the challenges posed by the plethora of query languages currently in use.

Asaf and I founded Logz.io in 2015 to provide developers with the ultimate open source log management experience. With our product, logging with the ELK Stack was simple, efficient, and automated for the first time – so customers could save engineering costs and accelerate MTTR.

Logz.io is honored to have our Cloud SIEM as one of the products available as part of the launch of AWS AppFabric. For customers invested in AWS, this inclusion allows you to use our cloud-based, agile SIEM seamlessly alongside other critical SaaS applications.

Over the last decade, log management has been largely dominated by the ELK Stack – a once-open source tool set that collects, processes, stores and analyzes log data. The ‘k’ in the ELK Stack represents Kibana, which is the component engineers use to query and visualize their log data stored in Elasticsearch. Sadly, in January 2021, Elastic decided to close source the ELK Stack, and as a result, OpenSearch was launched by AWS as an open source replacement.

OpenTelemetry (also abbreviated as OTEL) is an increasingly popular open-source observability platform under the Cloud Native Computing Foundation (CNCF), which is currently the most active project in the CNCF after Kubernetes. It was created to establish a unified and vendor-agnostic way for instrumenting, collecting, and exporting telemetry data for your system and application across traces, logs, and metrics.

The basic goal of log management is to make log data easy to locate and understand so that users can identify how their services are performing and troubleshoot more quickly. Logging as a Service, or LaaS, takes log management a step further by providing a solution that seamlessly scales and manages your log data via cloud-native architecture.

Today’s IT systems are ever more fragmented. It is commonplace to see polyglot systems, written in multiple programming languages, and using a plethora of tools and cloud services as infrastructure building blocks, whether data stores, web proxy or other functions. In this dynamic cloud-native realm, open standards and open specifications have become integral drivers of compatibility, collaboration, and convergence – the Three C’s of Open Standards, if you will.

SIEM has been a crucial component of security systems for nearly two decades. While there’s ample information on operating SIEM solutions out there, guidance on evaluating and managing them effectively is lacking. We’ve noticed many SIEM vendors are taking advantage of this dearth of knowledge and not providing customers with needed value for what they’re buying.

While log parsing isn’t very sexy and never gets much credit, it is fundamental to productive and centralized log analysis. Log parsing extracts information in your logs and organizes them into fields. Without well-structured fields in your logs, searching and visualizing your log data is near impossible.