In the context of logging, multiline logs happen when a single log is written as multiple lines in the log file. When logs are sent to 3rd party log monitoring platforms like Coralogix using standard shipping methods (e.g. Fluentd, Filebeat), which read log files line-by-line, every new line creates a new log entry, making these logs unreadable for the user.

Out of the four basic computing resources (storage, memory, compute, network), storage tends to be positioned as the foremost one to focus on for any architect optimizing an Elasticsearch cluster. Let’s take a closer look at a couple of interesting aspects in relation to the Elasticsearch storage optimization and let’s do some hands-on tests along the way to get actionable insights. The storage topic consists of two general perspectives.

Elasticsearch provides a powerful set of options for querying documents for various use cases so it’s useful to know which query to apply to a specific case. The following is a hands-on tutorial to help you take advantage of the most important queries that Elasticsearch has to offer. In this guide, you’ll learn 42 popular query examples with detailed explanations, but before we get started, here’s a summary of what the types of queries we’ll tackle.

It was a cloudy winter morning when I had arrived at the office and found, to our horror, that a Kubernetes cluster was suffering from extremely high CPU and network usage and had become almost completely non-functional. To make things worse, restarting the nodes (the go-to DevOp solution), seemed to have absolutely no effect on the issue. Something was poisoning the network and we had to find out what it was and fast.

The evolution of Security Information and Event Management (SIEM) is deeply intertwined with cloud computing, both in terms of technological breakthroughs the cloud provided and from its inherent security challenges. With the rise of cloud computing, we no longer rely on long-lived resources. An ephemeral infrastructure obscures the identity of the components and, even if you do have the visibility it doesn’t necessarily mean you can comprehend the meaning behind the components.

The power and value that’s embedded in logs are reflected by the status and behavior of our applications and infrastructure. Many times we would like to be alerted when the application or its components show abnormal behavior. This behavior can be reflected by the application sending some logs at a higher than usual volume. Figuring out exactly what ‘higher than usual’ means, or in other words, setting the threshold value at which the alert should trigger can be a daunting task.

10 years ago, Google was facing a critical bottleneck caused by extremely prolonged C++ compilation times and needed a totally new way to solve it. Google engineers tackled the challenge by creating a new language called Go (aka Golang). The new language of Go borrows the best parts of C++, (most notably it’s performance and security features) and combines it with the speed of Python to enable Go to quickly use multiple cores while being able to implement concurrency.

Amazon ELB (Elastic Load Balancing) allows you to make your applications highly available by using health checks and intelligently distributing traffic across a number of instances. It distributes incoming application traffic across multiple targets, such as Amazon EC2 instances, containers, IP addresses, and Lambda functions. You might have heard the terms, CLB, ALB, and NLB. All of them are types of load balancers under the ELB umbrella.

So, you’ve installed Coralogix’s STA and you would like to start analyzing your traffic and getting valuable insights but you’re not sure that you’re mirroring enough traffic or wondering if you might be mirroring too much data and could be getting more for less. The harsh truth is that in order to be able to detect everything, you have to capture everything and in order to be able to investigate security issues thoroughly, you need to capture every network packet.

Beats are lightweight, purpose-built agents that acquire data and then feed it to Elasticsearch. Beats use the libbeat framework that makes it easy to create customized beats for any type of data you’d like to send to Elasticsearch. Auditbeat is a lightweight shipper from the Beats family that you can install on your servers to audit the activities of users and processes on your systems.