Operations | Monitoring | ITSM | DevOps | Cloud

Sometimes silence isn’t golden; it’s a red flag. Whether you’re monitoring IoT sensors, system logs, or application metrics, missing data can be just as critical as abnormal data. Without visibility into these gaps, you risk overlooking potential failures, security threats, or operational inefficiencies. In time series workflows, detecting silence is often the first sign of trouble—whether it’s a network issue, device failure, sensor failure, or stalled process.

Part 2: Optimizers in Apache DataFusion In the first part of this post, we discussed what a Query Optimizer is and what role it plays and described how industrial optimizers are organized. In this second post, we describe various optimizations found in Apache DataFusion and other industrial systems in more detail.

As your time series data grows, managing increasing workloads can quickly become a headache. High data ingestion rates, numerous (and complex) queries, intensive processing tasks, and routine maintenance like data compaction often compete for limited resources. This leads to unpredictable performance and slower response times, and common solutions often introduce operational complexity.

Sometimes Query Optimizers are seen as a sort of black magic, “the most challenging problem in computer science,” according to Father Pavlo, or some behind-the-scenes player. We believe this perception is because: However, Query Optimizers are no more complicated in theory or practice than other parts of a database system, as we will argue in a series of posts: Part 1: Part 2: After reading these blogs, we hope people will use DataFusion to.

A common problem in monitoring and alerting systems is not just alerting on what you’re seeing but preventing alert storms from overwhelming operators. When a system generates multiple notifications for the same incident, it leads to alert fatigue and can mask other important issues. For time series data, alert fatigue can result in missed anomalies, delayed responses to critical trends, and difficulty distinguishing real performance degradations from noise.

In Industrial IoT for real-time monitoring, timely alerts are crucial. While Slack and email notifications are common, they can be easily missed or buried in a flood of other notifications. SMS and WhatsApp on the other hand, offer a level of immediacy and directness that’s hard to ignore.

InfluxDB 3 Core and Enterprise introduce a powerful new diskless architecture that lets you store your time series data in cloud object storage while running the database engine locally. This approach offers significant advantages: you get the performance of a local database combined with the durability, scalability, and cost-effectiveness of cloud storage. In this tutorial, I’ll show you how to set up InfluxDB 3 Core or Enterprise with Azure Blob Storage as your object store.

Today we’re excited to announce that InfluxDB 3 Core, our new open source product licensed under MIT/Apache 2, and InfluxDB 3 Enterprise are now in beta. InfluxDB 3 Core is a high-speed, recent-data engine that collects and processes data in real-time, while persisting it to local disk or object storage. InfluxDB 3 Enterprise is a commercial product that builds on Core’s foundation, adding high availability, read replicas, enhanced security, and data compaction for faster queries.

Monitoring is only as good as the alerts that surface critical issues before they spiral out of control. With InfluxDB 3 Core and Enterprise, you can extend alerting capabilities beyond built-in solutions by leveraging custom Python processing plugins. Whether you need real-time notifications when thresholds are exceeded or advanced anomaly detection tailored to your infrastructure, developing custom alerting logic ensures you get the right alerts at the right time.

One of the most compelling features of InfluxDB 3 is its built-in Python Processing Engine, a versatile component that adds powerful, real-time processing capabilities to both InfluxDB 3 Core and Enterprise. For those familiar with Kapacitor in InfluxDB 1.x or Flux Tasks in 2.x, the Processing Engine represents a more streamlined, integrated, and scalable approach to acting on data.