For many companies today business success depends on quality efficient build, test and delivery processes. Development and deployment environments become complex very quickly, even for small and medium companies. A contributing factor to this complexity is companies’ adoption of microservices. This is where modern CI/CD solutions like CircleCI come in to help streamline these processes.
In this post we will cover some of the main use cases FluentD supports and provide example FluentD configurations for the different cases.
Are you building and deploying software manually and would like to change that? Are you interested in learning about building a Jenkins pipeline and better understand CI/CD and DevOps at the same time? In this first post, we will go over the fundamentals of how to design pipelines and how to implement them in Jenkins. Automation is the key to eliminating manual tasks and to reducing the number of errors while building, testing and deploying software.
In the context of logging, multiline logs happen when a single log is written as multiple lines in the log file. When logs are sent to 3rd party log monitoring platforms like Coralogix using standard shipping methods (e.g. Fluentd, Filebeat), which read log files line-by-line, every new line creates a new log entry, making these logs unreadable for the user.
Out of the four basic computing resources (storage, memory, compute, network), storage tends to be positioned as the foremost one to focus on for any architect optimizing an Elasticsearch cluster. Let’s take a closer look at a couple of interesting aspects in relation to the Elasticsearch storage optimization and let’s do some hands-on tests along the way to get actionable insights. The storage topic consists of two general perspectives.
Elasticsearch provides a powerful set of options for querying documents for various use cases so it’s useful to know which query to apply to a specific case. The following is a hands-on tutorial to help you take advantage of the most important queries that Elasticsearch has to offer. In this guide, you’ll learn 42 popular query examples with detailed explanations, but before we get started, here’s a summary of what the types of queries we’ll tackle.
It was a cloudy winter morning when I had arrived at the office and found, to our horror, that a Kubernetes cluster was suffering from extremely high CPU and network usage and had become almost completely non-functional. To make things worse, restarting the nodes (the go-to DevOp solution), seemed to have absolutely no effect on the issue. Something was poisoning the network and we had to find out what it was and fast.
The evolution of Security Information and Event Management (SIEM) is deeply intertwined with cloud computing, both in terms of technological breakthroughs the cloud provided and from its inherent security challenges. With the rise of cloud computing, we no longer rely on long-lived resources. An ephemeral infrastructure obscures the identity of the components and, even if you do have the visibility it doesn’t necessarily mean you can comprehend the meaning behind the components.
The power and value that’s embedded in logs are reflected by the status and behavior of our applications and infrastructure. Many times we would like to be alerted when the application or its components show abnormal behavior. This behavior can be reflected by the application sending some logs at a higher than usual volume. Figuring out exactly what ‘higher than usual’ means, or in other words, setting the threshold value at which the alert should trigger can be a daunting task.
10 years ago, Google was facing a critical bottleneck caused by extremely prolonged C++ compilation times and needed a totally new way to solve it. Google engineers tackled the challenge by creating a new language called Go (aka Golang). The new language of Go borrows the best parts of C++, (most notably it’s performance and security features) and combines it with the speed of Python to enable Go to quickly use multiple cores while being able to implement concurrency.
