Before the need for log correlation, there was a time not so long ago when reading software application logs was simple. Your application would output log files in sequential order, and you’d read through them. In the event of a bug, software outage, or security incident, you could easily parse what happened and when. It was a tedious process, but it was simple.
Following the fully featured public beta of the Elasticsearch Service on Microsoft Azure earlier this year, we are pleased to declare it is now generally available! Existing Elasticsearch Service customers can log in and launch deployments on Azure in their existing accounts, and new users can get started with a free 14-day trial of the Elasticsearch Service.
Behavior analytics is a powerful tool in intrusion detection and prevention, using insights from typical user behavior to detect suspicious activity. What role does log monitoring have in this process and how does it work together with behavioral analytics?
Kubernetes has become the de-facto solution for container orchestration. While it has, in some ways, simplified the management and deployment of your distributed applications and services, it has also introduced new levels of complexity. When maintaining a Kubernetes cluster, one must be mindful of all the different abstractions in its ecosystem and how the various pieces and layers interact with each other in order to avoid failed deployments, resource exhaustion, and application crashes.
With the addition of new data structures in Lucene 6.0, the Elasticsearch 5.0 release delivered massive indexing and search performance improvements for one-dimension numeric, date, and IP fields, and two-dimension (lat, lon) geo_point fields. Building on this work, the Elasticsearch 6.0 release further improved usability and simplicity of the geo_point API by setting the default indexing structure to the new block k-d tree (BKD) and removing all support for legacy prefix tree encoding.
Software giant Adobe is known the world around for its Photoshop, Illustrator, and Acrobat products, which are rolled into cloud service suites — Creative Cloud, Document Cloud, and Experience Cloud — of other similar software offerings. A number of their products — especially those where image search is critical, such as Adobe Stock — feature slick search capabilities that use Elasticsearch behind the scenes.
This was a particularly interesting and challenging year for security, cloud computing, and DevOps. It has been exciting to watch organizations and individuals transition to the public cloud and adopt DevOps culture and processes. Many are now reaping the benefits of these technological shifts.
As part of Elasticsearch 7.5.0, a new ingest processor — named enrich processor — was released. This new processor allows ingest node to enrich documents being ingested with additional data from reference data sets. This opens up a new world of possibilities for ingest nodes.
If you are reading this, I don’t have to convince you any further of the powerful intelligence we can derive from logs and machine data. If you are anything like the many, many users, customers and prospects we have been talking to over the years, you might, however, have some level of that pesky modern condition commonly known as volume anxiety. The volume here, of course, is the volume of data––there is a lot of it, and it keeps growing.
