Operations | Monitoring | ITSM | DevOps | Cloud

Elastic Cloud on Kubernetes (ECK) is an operator that allows you to automate the deployment of the Elastic Stack — including Elasticsearch, Kibana, and Elastic APM, Elastic SIEM, and more — using Kubernetes. By using this ECK, you can quickly and easily deploy Elasticsearch clusters with Kubernetes, as well as secure and upgrade your Elasticsearch clusters. It is the only official Elasticsearch operator.

When talking about log management, search history is overlooked more often than not. Past searches can be used as part of log analysis and forensic analysis, but the main issue with this data is the speed of search which gets compromised as data volume gets greater. We will discuss some ways to get the best out of your saved searches and to speed up the search process.

Beats are lightweight, purpose-built agents that acquire data and then feed it to Elasticsearch. Beats use the libbeat framework that makes it easy to create customized beats for any type of data you’d like to send to Elasticsearch. Auditbeat is a lightweight shipper from the Beats family that you can install on your servers to audit the activities of users and processes on your systems.

Logs from a variety of different AWS services can be stored in S3 buckets, like S3 server access logs, ELB access logs, CloudWatch logs, and VPC flow logs. S3 server access logs, for example, provide detailed records for the requests that are made to a bucket. This is very useful information, but unfortunately, AWS creates multiple .txt files for multiple operations, making it difficult to see exactly what operations are recorded in the log files without opening every single .txt file separately.

In the olden days, we used to have to get logs by putting our agent on one machine at a time, like hitching a horse to a horse-drawn carriage. But now, we’ve got Kubernetes. It’s like a horse factory, and we’ve got more horses than we know what to do with. In this wild west of containerization, we could quickly end up underneath more logs than our old-timey agent could keep track of! But now there’s a new sheriff in town.

AWS has a lot of services, and they all generate logs. A lot of logs. We’ve worked hard to make sure you can capture logs from every source and service on AWS, and today we’re happy to announce the final piece of our AWS logging puzzle: LogDNA’s S3 Collector integration. It’s an easy-to-use Lambda function that lets you ingest any AWS logs that get dumped to S3 – like logs from CloudFront and ELB.

This is the one post I hope you’ll never need. However, should you ever need it, this is your one-stop shop for understanding how to proceed with DevOps incident management. Have you just been attacked? Did the commit go wrong? A CI pipeline went haywire? Don’t worry. I got you.

As a company leader, what value are you extracting from your business and IT? What about from security data lakes and processes? Are you able to drive speed for your business because of real-time, advanced analytic capabilities? Successful companies are making technology architecture their business architecture—aligning stakeholders, processes, data, and analytics in an integrated fashion to deliver faster, data-driven decisions that are more accurate than ever before.

During their time at theCUBE, some of our executives talk about what Data-to-Everything really means, and how our latest product releases help organizations bring data to more questions, decisions and actions. Here are their insights.