SEMplicity: Scaling Large ECE Deployments

SEMplicity: Scaling Large ECE Deployments

Apr 17, 2020

From the trenches: what does it really take to scale up a large Elastic security log deployment?

Elasticsearch for enterprise security log storage & management is a hot topic today. Specular gains in performance, functionality and cost are ready for harvest. But what exactly does it take to create a large Elastic log storage infrastructure? This talk will present war stories related to at 150,000 events per second Elastic log storage implementation with 2 month retention built at a large commercial client. We'll take the audience through sizing, design. staffing & cost; discuss architecture, storage density & ingestion: and share our gotchas & lessons learned. We will also talk a bit about evidentiary-quality log storage for compliance. If you are curious about what it would take for Elastic to hold your security logs, this talk will show you what to expect.