Operations | Monitoring | ITSM | DevOps | Cloud

On your bedside table sits a piece of software designed to wake you up. It loves bothering you when something goes wrong — and making it your responsibility to sort it out Meet the new incident.io On-call app. We designed it this way: to be as interruptive as possible. Whether you’re watching telly, at the gym, or as mentioned, fast asleep, it’ll get you. Got called even though you’re in silent mode? Great! We’ve done our job properly.

The on-call experience is sometimes a dreaded one for software engineers. Those late-night alerts and frantic Slack messages, after all, don’t exactly sound pleasant. But what’s an on-call shift really like? Is that perception of constant fire-fighting and 3 AM wake-up calls actually realistic? Michael Mandrus and Owen Smallwood, both senior software engineers here at Grafana Labs, wanted to set the record straight.

ITOps and DevOps teams face many challenges. Their responsibilities are extensive, from navigating complex IT environments at scale to quickly addressing performance issues and minimizing downtime and outages. Enhancing your organization’s IT service assurance requires you to ensure the reliability, performance, and availability of IT services.

Everyone experiences stress at work—thankfully, it’s a topic folks aren’t shying away from anymore. But for on-call engineers, alert fatigue is a phenomenon closer to home. Unfortunately, like stress, it can be just as insidious and drastically impact those it affects. First discussed in the context of hospital settings, this phrase later entered engineering circles.

Today, incident management is a core part of organizations both big and small. But what if you don't have a program in place...where do you start? Or what if incident management is already a key part of your org, but you're looking to optimize it—where do you kick things off in that case? Consider another situation: What if you're an established organization with years of incident management experience—what are some things that you can do to take things to the next level?

Incidents are a stressful time for your team: your service isn't working the way you expect and your customers/stakeholders want to know what's going on. The last thing you want to do is let your team improvise everything when it comes to responding to incidents. Google's own SRE book has great overall tips for incident management, part of which involves "develop(ing) and document(ing) your incident management procedures in advance", which this article dives into.

On this weeks' episode of The Debrief, we chatted with Jeff Forde, an Architect on the Platform Engineering team at Collectors. With a background spanning finance, healthcare, and various product-led startups, Forde has honed his expertise in DevOps, site reliability, and platform engineering. Beyond his professional life, he's also a dedicated volunteer first responder and certified fire instructor in Connecticut, offering him a unique perspective on managing incidents of all typesz.

Say hello to smarter incident management with smart summaries, mitigation message suggestions, and our new conversational assistant! 🚀✨