In the world of Software Engineering, observability seems to be the talk of the town. We discuss it at conferences, read about it in blogs or articles, and see it promised to us by vendor after vendor. But what is observability? What issues have recently evolved to make it such an integral concept? What strategies are engineers employing to ensure observability? And most importantly of all, why are engineers looking to achieve it?

Serving over 44% of the world’s websites, Apache is by far the most popular web server used today. Apache, aka Apache HTTP Server, aka Apache HTTPd, owes its popularity to its ease of use and open-source nature but also its inherent flexibility that allows engineers to extend Apache’s core functionality to suit specific needs.

Apache Flume helps organizations stream large log files from various sources to distributed data storage like Hadoop HDFS. This article focuses on the features and capabilities of Apache Flume and how it can help applications efficiently process data for reporting and analytical purposes.

Even though the public cloud has become a commonplace concept in today’s tech world, many companies in a variety of industries remain slow to leverage its advantages. One commonly cited reason for this is that even for experienced IT experts, the transition to and adoption of cloud technologies can be daunting.

Load balancing, traffic management, authentication and authorization, service discovery — these are just some of the interactions taking place between microservices. Collectively called a “service mesh”, these interconnections can become an operations headache when handling large‑scale, complex applications. Istio seeks to reduce this complexity by providing engineers with an easy way to manage a service mesh.

Monitoring is an essential aspect of any IT system. System metrics such as CPU, RAM, disk usage, and network throughput are the basic building blocks of a monitoring setup. Nowadays, they are often supplemented by higher-level metrics that measure the performance of the application (or microservice) itself as seen by its users (human beings on the internet or other microservices in the same or different clusters).

As discussed in our blog post [What is User Activity Monitoring?], user activity monitoring (UAM) is a form of surveillance that provides visibility and insight into employee productivity and engagement while also revealing insider security threats. While UAM on company-owned or company-sanctioned devices and networks is legal, ethical and HR considerations require that UAM be implemented with a high level of professionalism and sensitivity.

User Activity Monitoring (UAM) tracks the behavior of internal end-users—employees, subcontractors, partners, and so on—on a company’s networks, devices, and other IT resources. UAM, sometimes also called employee monitoring, may be deployed for a number of reasons, such as providing insight into the productivity of both individual employees and the company as a whole. Is Employee X spending too much time browsing the internet for non-business purposes during work time?

In a previous post, I explained the role Apache Kafka plays in production-grade ELK deployments, as a message broker and a transport layer deployed in front of Logstash. As I mentioned in that piece, Redis is another common option. I recently found out that it is even more popular than Kafka! Known for its flexibility, performance and wide language support, Redis is used both as a database and cache but also as a message broker.

DevOps, security, and SOC teams find themselves constantly facing new cyber threats, ever-evolving attackers, and innovative attack vectors. Their challenges range from protecting employees’ mobile devices to preventing malicious parties from accessing an organization’s financial data or customers’ personal information.

Open source is one of the key drivers of DevOps. The need for flexibility, speed, and cost-efficiency, is pushing organizations to embrace an open source-first approach when designing and implementing the DevOps lifecycle. Monitoring — the process of gathering telemetry data on the operation of an IT environment to gauge performance and troubleshoot issues — is a perfect example of how open source acts as both a driver and enabler of DevOps methodologies.

IoT, or the Internet of Things, has made its way into every corner of our lives. Once upon a time, the idea of an inescapable internet may have seemed like a far-off dream. Today, it’s our reality. Internet connected devices are everywhere—from our fitness trackers to our vehicles and appliances. These devices track our sleep patterns, enable us to set our coffee machines remotely, and find our pets after they have wandered off, among countless other tasks.

One of the benefits of using Logstash in your data pipelines is the ability to transform the data into the desired format according to the needs of your system and organization. There are many ways of transforming data in Logstash, one of them is using the mutate filter plugin. This Logstash filter plugin allows you to force fields into specific data types and add, copy, and update specific fields to make them compatible across the environment.

Kubernetes is the de facto industry standard for container management and orchestration. Not surprisingly, it has also become common to use Kubernetes in tandem with compatible Continuous Integration (CI) and Continuous Deployment (CD) tools. As a container orchestrator, Kubernetes expects deployments to spin up software throughout a cluster. Those deployments are created using files or command lines that can be generated manually or using properly configured CI/CD software.

Over the past few years, and coupled with the growing adoption of microservices, distributed tracing has emerged as one of the most commonly used monitoring and troubleshooting methodologies. New tracing tools and frameworks are increasingly being introduced, driving adoption even further. One of these tools is Jaeger, a popular open source tracing tool. This article explores the integration of Jaeger with the ELK Stack for analysis and visualization of traces.

Apache Tomcat is the most popular application server for serving Java applications. Widely-used, mature and well documented, Tomcat can probably be defined as the de-facto industry standard. Some sources put Tomcat’s market share at over 60%! Tomcat is particularly popular for serving smaller applications since it doesn’t require the full Java EE platform. It consumes a relatively small amount of resources and provides users with simpler admin features.