Operations | Monitoring | ITSM | DevOps | Cloud

In this blog, we’re going to take a look at how you can set up a fully-functioning, robust data pipeline to centralize your data into an InfluxDB instance by collecting and sending messages with the MQTT protocol. We’ll start with a brief overview of the technologies and protocols used in the pipeline, then dive into how you can connect, configure, and test them to ensure your data pipeline is fully functional. It’s going to be a long post, so let’s jump right in.

Last month I was making a change to sx, our CLI. I updated a core flow, adding external catalogs as a source for sx add. Small change. Then came the testing. I knew I was messing with a core flow and wanted to be sure I hadn't broken anything. I spent about forty-five minutes setting up an isolated environment. Spinning up Docker. Fighting with tmux. Getting a clean install state I could run through the TUI a few times. Forty-five minutes of my afternoon that produced zero code. I complained in Slack.

Healthchecks.io ping endpoints accept HTTP HEAD, GET, and POST request methods. When using HTTP POST, clients can include an arbitrary payload in the request body. Healthchecks.io stores the first 100kB of the request body. If the request body is tiny, Healthchecks.io stores it in the PostgreSQL database. Otherwise, it stores it in S3-compatible object storage. We recently migrated from a managed to a self-hosted object storage.