April was a month to remember at incident.io. Not only did we attend our second conference ever with KubeCon in Amsterdam, but we also very subtly released our brand-new Status Pages product. OK, it probably wasn't subtle. Both moments required months of preparation, feedback loops, iteration, and so much more behind-the-scenes work to get right. So if you ran into us at KubeCon, thank you for stopping by and meeting with our team.

Incident management is a team effort. While it's true that incident management should be seen as a company-wide effort, and you should empower all teams to declare incidents, this differs from the team effort I'm referring to here. No, incident management is a team effort in the sense that no one tool can do it all, not even incident.io. We covered as much when we discussed why we integrate with tools that can be seen as our competitors – and that’s OK!

Earlier this year, we experienced intermittent timeouts in our application while interacting with our database over a period of two weeks. Despite our best efforts, we couldn’t immediately identify a clear cause; there were no code changes that significantly altered our database usage, no sudden changes in traffic, and nothing alarming in our logs, traces, or dashboards. During that two-week period, we deployed 24 different performance and observability-focused changes to address the problem.

We kicked off 2023 with a new team and a new product to build - Status Pages. We wanted to build a solution we could ship to customers as quickly as possible, while making sure that it’s reliable, fast and beautiful. Here’s how that process played out over the course of three months.

Clear and frequent communication carries considerable weight in today's era of hyper-competition among businesses—especially during incidents. Because of this, status pages have become the go-to choice for companies looking to prioritize trust, transparency, and clarity with their customers, even during downtime. Unfortunately, current status page solutions have made these communications particularly frustrating and stressful.

Communication carries more weight than ever before. Businesses are so much more connected to their customers given the number of mediums they can communicate through; Twitter, Instagram, Facebook, and even TikTok. Because of this, it's essential to prioritize these lines of communication throughout your day-to-day. Some might even say that over-communicating is the best way forward. Why? No one likes a company that appears simply like a black box with zero insight into what's happening.

I’m a big fan of monolithic architectures. Writing code is hard enough without each function call requiring a network request, and that’s before considering the investment in observability, RPC frameworks, and dev environments you need to be productive in a microservice environment.

At Vanta, our goal is to nurture a positive security culture in everything we do—which is especially critical given that helping our customers improve their security and compliance posture starts with our own. Employees are the key to our security resilience, so we strive to build and support a strong culture of incident response in tandem. Here’s what that means to us at Vanta.

Cattle, not pets is a DevOps phrase referring to servers that are disposable and automatically replaced (cattle) as opposed to indispensable and manually managed (pets). Local development environments should be treated the same way, and your tooling should make that as easy as possible. Here, I’ll walk through an example from one of my first projects at incident.io, where I reset my local environment a few times to keep us moving quickly.

Think about this—what was the last incident that challenged you? Did you learn anything from it? It will be shocking to no one to hear that we deal with our fair share of incidents. These run the gamut from tiny bugs to significant outages (thankfully, the latter happening only very rarely 😮‍💨). Either way, we always take the time to learn from them in some way. This might look like changes to our response processes or revisiting systems we’re using.