This article explores integrating Google Pub/Sub with the world’s most popular open source log analysis platform — the ELK Stack, for deeper analysis and investigation.

We live in a world of big data, where even small-sized IT environments are generating vast amounts of data. Once an organization has figured out how to tap into the various data sources generating the data, and the method for collecting, processing and storing it, the next step is analysis.

Once you have your Elasticsearch running, you’ll likely eventually find that performance starts to suffer over time. This can be due to a variety of factors, including changes in the way you’re using your cluster to how much and what types of data are being sent in. In order to maintain your cluster, you’ll need to set up monitors to alert you to any warning signs so that you can proactively handle available maintenance windows.

We are happy to inform our users that a new Docker logging plugin is available on the Docker Store! Using this plugin, users can easily ship container logs directly to Logz.io, and enjoy the following benefits.

In the container world of today, two words rule in bliss — Docker and Kubernetes. Two extremely popular platforms used for managing containers that for a beginner at least, may also seem to be competing technologies.

While Logz.io provides Kibana — the ELK Stack’s visualization tool — as part of its service, a lot of users have asked us to support Grafana. One of the leading open source visualization tools today, Grafana has some added value when compared to Kibana, especially around visualizing time-series data.

We’ve all been there — you’re on-call, fast asleep at 3 AM when suddenly, in comes the alerts–in overdrive. Your system is notifying you of some sort of abnormal behavior, but with all the alerts and data coming through, its difficult to figure out what your system is trying to tell you. Is there potential malicious behavior? Did someone write faulty code? Is it an important issue or can it wait? Is it nothing at all?

What the Beats family of log shippers are to Logstash, so Fluent Bit is to Fluentd — a lightweight log collector, that can be installed as an agent on edge servers in a logging architecture, shipping to a selection of output destinations.

While logs can tell us whether a specific request failed to execute or not and metrics can help us monitor how many times this request failed and how long the failed request took, traces help us debug the reason why the request failed, or took so long to execute by breaking up the execution flow and dissecting it into smaller events.

Open source has come a long way. Open source components are the building blocks of arguably every organization’s software. According to Stack Overflow’s 2018 developer survey results, nearly half of professional developers contribute to open source projects, and 40% listed contribution to open source software as part of their non-formal learning background.