Operations | Monitoring | ITSM | DevOps | Cloud

Logs from a variety of different AWS services can be stored in S3 buckets, like S3 server access logs, ELB access logs, CloudWatch logs, and VPC flow logs. S3 server access logs, for example, provide detailed records for the requests that are made to a bucket. This is very useful information, but unfortunately, AWS creates multiple .txt files for multiple operations, making it difficult to see exactly what operations are recorded in the log files without opening every single .txt file separately.

In the olden days, we used to have to get logs by putting our agent on one machine at a time, like hitching a horse to a horse-drawn carriage. But now, we’ve got Kubernetes. It’s like a horse factory, and we’ve got more horses than we know what to do with. In this wild west of containerization, we could quickly end up underneath more logs than our old-timey agent could keep track of! But now there’s a new sheriff in town.

It’s been a while since I have had the pleasure of announcing a new version of Security Monitoring (September 2018), but today I am doing just that. There is nothing better to inspire spending your evenings coding and playing with Splunk than your partner watching shows that just don’t interest you! For my UK friends, yes ‘Love Island’ is that show and for my more international friends "look it up!". So, what updates did I bring?

During their time at theCUBE, some of our executives talk about what Data-to-Everything really means, and how our latest product releases help organizations bring data to more questions, decisions and actions. Here are their insights.

Generally, 2009 to 2019 were 10 golden years for manufacturers worldwide. After the swift recovery from the economic and financial crisis in 2008/09, many manufacturers have been enjoying double-digit order intake growth, increasing revenues and profits for over a decade. German manufacturers in particular benefitted from an unprecedented peak in 2018. Volkswagen delivered a record-high number of 11 million cars and grew its revenues to 236 billion euros, the highest revenue ever in its long history.

Elasticsearch cross-cluster replication (CCR) was released as a beta feature in Elasticsearch 6.5, and as a Generally Available (GA) feature in Elasticsearch 6.7. CCR allows multiple indices to be replicated to one or more Elasticsearch clusters. Replicating indices to additional Elasticsearch clusters solves several use cases, including high availability (HA) across datacenters, disaster recovery (DR), and CDN-like architectures to co-locate data closer to application servers (and users).

Welcome to 2020! (We’re a little slow with that on the Loki team.) To kick off the year we are releasing Loki 1.3! Anyone running Loki in microservices mode will be excited by this release as it introduces the Loki Query Frontend. (If you aren’t using microservices, be patient – good things will be coming your way soon.) The query frontend sits in front of the queriers and allows sharding queries based on time.

AWS S3 Server Access logs provide detailed records for requests made to S3 buckets. They’re useful for many applications. For example, access log information can be useful in security and access audits. It can also help generate customer insights and better understand your Amazon S3 bill. Coralogix makes it easy to integrate with your S3 server access logs via a Lambda function.