Operations | Monitoring | ITSM | DevOps | Cloud

A Postgres database is one of the most complex pieces of software you run. It stores your data durably so a power loss doesn't lose a committed transaction. It hands out real transactions while thousands of queries fight over the same rows. It stays available when a node dies, lets you rewind to a point in time after someone drops the wrong table, and constantly tunes itself with background jobs. Two questions matter about all of that machinery: is it healthy right now, and will it stay healthy?

We run Postgres under an operator for automatic failover. That is a promise about what happens during a failure, so the only way to know you have it is to cause the failure and watch. The docs tell you what should happen. A config review tells you which knobs are set. Neither tells you how long an isolated primary keeps accepting writes after its replacement has been promoted, and that number decides whether a failover is clean or leaves you with two versions of your data.

I wanted a Postgres replica that falls behind its primary. Sounds easy. It wasn't. Every time I cut the replica off from the primary, its lag jumped for a moment and then dropped right back to zero. It just would not stay behind. Chasing down why turned into a fun little tour of how a Postgres standby keeps itself alive, and what CloudNativePG quietly sets up behind the scenes.

Autovacuum is one of those Postgres background jobs that quietly keeps your database healthy. It cleans up the dead row versions that every UPDATE and DELETE leaves behind, and it keeps the database away from a hard transaction-ID limit that would take it offline. Most of the time you don't think about it, because it just works.