Operations | Monitoring | ITSM | DevOps | Cloud

Elasticsearch has long been the backbone of security analytics for organizations that need fast search, flexible dashboards, and scalable visibility across massive datasets. It powers everything from threat hunting to compliance reporting and real-time investigation. But anyone who has operated Elasticsearch at scale also knows a quiet truth: Elasticsearch is only as strong as the data you feed it. And getting clean, consistent, usable telemetry into Elastic is often the hardest part.

This month’s DataStream update brings meaningful improvements across pipeline management, MSSP workflows, and endpoint visibility. We’ve focused on giving security teams more control over how data moves through their environment, expanding coverage for both Windows and Linux, and strengthening governance for multi-tenant deployments. Let’s walk through what’s new.

As security data volumes grow, so do the costs of processing and storing them. Microsoft Sentinel and other SIEM platforms charge based on data ingestion, which makes every decision about normalization rules critical and every duplicate log a direct expense. Enterprise-scale security data pipelines face a persistent problem: data duplication across normalization tiers. As logs move through multiple transformation stages, it’s often impossible to know in advance which version will succeed.

This update delivers significant advances in operational flexibility and security monitoring capabilities. It addresses the evolving needs of security teams across diverse deployment environments, from air-gapped networks to those prioritizing automation and simplicity, while expanding integration options and improving visibility into data flows.

Modern security operations teams face an overwhelming challenge: a rapidly growing volume of logs, alerts, and telemetry from cloud services, on-premises infrastructure, and third-party security tools. Traditional SIEM platforms often struggle to scale cost-effectively and provide the agility needed for advanced analytics and threat hunting.

Logstash has been a workhorse in data processing pipelines for years, but it was not designed with today’s security operations in mind. Security teams now deal with massive telemetry volumes, rising SIEM costs, and diverse log formats that require constant normalization. In this environment, Logstash shows its age: manual configuration, outdated parsing, and scalability bottlenecks introduce fragility instead of efficiency.

This month we’re rolling out several new capabilities designed to simplify daily work for SOC teams, MSSPs, and enterprise security operations. The focus is on easier access control, streamlined log management, and faster vendor onboarding.

Collecting Windows Security Events has always been a necessary but difficult job. Traditional methods depend on third-party collectors that must be installed, configured, and constantly maintained. They break, they lag behind updates, and they create unnecessary operational work. At the same time, they often flood Microsoft Sentinel with redundant or irrelevant data, driving up costs and slowing down investigations.

A practical guide for IT professionals, DevOps, security teams, platform engineers, and anyone who’s dealing with logs. In contemporary distributed systems, telemetry data—logs, metrics, traces, and events—serves as the primary mechanism for understanding internal system behavior. However, as system complexity increases, so does the volume and heterogeneity of telemetry.

We’re constantly improving DataStream to make security data management simpler, smarter, and more efficient for modern SOCs. This latest update introduces new capabilities that bring even more visibility and flexibility to your telemetry pipelines. Let’s take a closer look at what’s new.