Operations | Monitoring | ITSM | DevOps | Cloud

The first moments of incident response can be among the most crucial, which in turn can also make them among the most stressful. There are many ways to ensure incidents are kicked off smoothly, but a recent focus of ours was to ensure they could be kicked off quickly. After all, the faster you're able to start mitigating your incident, the more successful you'll be!

Just as a red flag warns of imminent danger, incident severity levels in IT Service Management (ITSM) act as crucial indicators that alert organizations to potential problems. By understanding and leveraging them, businesses can swiftly and effectively respond to incidents, minimizing their impact on operations. In the dynamic business operations landscape, unexpected disruptions are an unavoidable reality.

When our CEO and co-founder Tomer Levy delivered his “Observability is Broken” presentation at last year’s AWS re:Invent, he highlighted numerous challenges faced by today’s organizations as they seek to advance their observability practices. Of the six individual points that he noted, two specifically dealt with the current shortage of available engineering expertise, with another two focused on data overload.

Although the causes and solutions for incidents vary widely, most incidents follow a similar timeline from declaration to resolution. We call the period of time it takes to move from one phase or milestone of an incident to the next cycle time.

We understand that everyone values a simple and straightforward approach when it comes to setting up schedules. We at Squadcast are fully aware of the difficulties involved in creating an on-call schedule from scratch or migrating it to a new platform. Hence we have come up with a blog to assist you in seamlessly setting up your on-call schedule using Squadcast. Our goal is to provide guidance and support to make the process as effortless as possible for you.

Prometheus is a favored open-source monitoring system that collects, stores, and queries metrics from various sources. In Prometheus, an exporter is a component that collects and exposes metrics in a format Prometheus can scrape. The Prometheus Blackbox Exporter is designed to monitor “black box” systems with internal workings that are not accessible by Prometheus. It sends HTTP, TCP, and ICMP requests to the external systems and measures their response times and statuses.

Before we dive into the nitty-gritty of incident management, let’s look a bit closer at the actual meaning of ‘incident.’ In the world of IT service management, the official definition for ‘incident’ is an “unplanned interruption to an IT service or reduction in the quality of an IT service.” Whether that means a slowdown in response time or a total system crash, you’re looking at an incident.

This week I’ve been reading through the recent judgment from the Swedish FSA on the Swedbank outage. If you’re unfamiliar with this story, Swedbank had a major outage in April 2022 that was caused by an unapproved change to their IT systems. It temporarily left nearly a million customers with incorrect balances, many of whom were unable to meet payments.

It feels great writing this. It's hard to believe that we have been working on Spike.sh full-time for 3 years now. It's been the most rewarding experience of my life. A big thank you to all of our users and your constant feedback, which has only made Spike.sh better month on month. We are - Over the years, we have always kept our heads down and built. During this entire process, we have learnt a huge deal of things when it comes to incidents and how they are being managed.