If you’re using Solr, for sure there are times when you change the schema and need to reindex. Quite often the source of truth is a database, so you can use streaming expressions via the JDBC source to reindex. But sometimes that’s not possible or adds too much load to the DB. So how can we use Solr itself as a source?
For this Black Friday, Sematext wishes you: Now seriously, applications tend to generate a lot more logs on Black Friday, and they also tend to break down more – making those logs even more precious. If you’re using the Elastic Stack for log management, in this post we’ll share some tips and tricks to prepare you for this extra traffic.
Apache Zookeeper is a great tool used by many popular tools. Your Kafka uses Zookeeper, your HDFS uses it, your SolrCloud uses it, and your ClickHouse may also be using it. No matter where you are using Apache Zookeeper, it is usually a crucial piece of the infrastructure and it needs to be reliable and fast.
No matter how well-designed, flashy, or useful your application is for your target users, they may not take kindly to it being slow or, even worse, crashing once in a while. You will lose customers and revenue as a result. The solution is definitely not to add additional features to the application to bring back users. Instead, it’s as simple as paying close attention to the health of the servers where your application is hosted.
PostgreSQL is an open-source relational database management system that’s been utilized in continuous development and production for 30 years now. Nearly all the big tech companies use PostgreSQL, as it is one of the most reliable, battle-tested relational database systems today. PostgreSQL is a critical point in your infrastructure, as it stores all of your data. This makes visibility mandatory, which in turn means you have to understand how logging works in PostgreSQL.
RabbitMQ is one of the most popular open-source message brokers available. Its ability to be deployed in various configurations and on various platforms makes it a widely used tool; it also supports all major messaging protocols, making it very versatile. Still, debugging issues with a tool like RabbitMQ can be challenging, especially when it’s deployed on a large cluster. RabbitMQ logs are one way to go, as they help you backtrack to an earlier point while debugging.
The software is getting more and more complicated and so is the infrastructure behind it. It is no longer what it used to be with a single web or application server and a database backing it up. Throughout the years, the infrastructure has become more and more complicated. We have multiple databases, queues, datastores, search engines, and configurations. We want to incorporate continuous delivery and automated testing and deploy everything easily.
NGINX is one of the most widely used reverse proxy servers, web servers, and load balancers. It has capabilities like TLS offloading, can do health checks for backends, and offers support for HTTP2, gRPC, WebSocket, and most TCP-based protocols. When running a tool like NGINX, which generally sits in front of your applications, it’s important to understand how to debug issues. And because you need to see the logs, you have to understand the different NGINX logging mechanisms.
The world lives by processing the data. Humans process the data – each sound we hear, each picture we see – everything is data for our brain. The same goes for modern applications and algorithms – the data is the fuel that allows them to function and provide useful features. Even though such thinking is not new, what is new in recent years is the requirement of near-real-time processing of large quantities of events processed by our systems.
Containers are no longer a thing of the future – they are all around us. Companies use them to run everything – from the simplest scripts to large applications. You create a container and run the same thing locally, in the test environment, in QA, and finally in production. A stateless box built with minimal requirements and unlike virtual machines – without the need of virtualizing the whole operating system.
