Operations | Monitoring | ITSM | DevOps | Cloud

Change one field in a.proto from double to string, and nothing complains. The code compiles, the deploy goes green, and a service still built from the old contract keeps answering with a double. Here’s what Go’s protobuf runtime does when a client on the new contract reads that answer: No error. risk_score comes back empty, and the nine bytes of the double are tucked away as unknown fields.

Replay a GraphQL recording at the server that produced it, and every request comes back 200 OK. Here is one of those 200s, from a Session query whose recorded token has expired: A GraphQL server reports most failures inside a successful HTTP response, so any check that stops at the status code passes this request. And that is the second way GraphQL hides from URL-shaped tooling.

A readiness check should make a release safer. During one release validation, ours kept Kraken’s traffic replays stuck in Initializing until they timed out. The operator was asking Kubernetes for EndpointSlices. Its installed permissions did not allow that request. The code needed a real cluster, the generated permissions, and a replay trying to start before that mistake became obvious. Kraken put those pieces together during release validation.

On October 20, a power failure in one of Google Cloud’s three London zones disrupted Revolut, Lloyds, HSBC and others for hours. One zone failed, errors spread region-wide, and Google warned customers they might not be able to create new workloads to fail over to. Most teams find out how resilient they are during an outage, not before one. These five questions let you find out first.