Have you ever been paged for a critical issue and started troubleshooting only to find an obvious drop in requests that weren’t caught by a static threshold? Or a significant increase in a metric that didn’t cross a static threshold? Or even, evidence of warning alerts triggered long ago that should have enabled someone to resolve the issue and prevent it from causing business impact, but instead was ignored in the massive alert volume received by the team?
As systems become increasingly complex, we’ve seen the growth of engineering tools to abstract away and manage the complexity. But often our tools are “opinionated” and the default actions or settings may not align with how our systems are intended to work or how we think they work. Chaos Engineering is a good way to not only test your applications, but also the tools you use to build them.
For decades, information technology in the financial services industry meant deploying bulky applications onto monolithic systems like mainframes. These systems have a proven track record of reliability, but don’t offer the flexibility and scalability of more modern architectures such as microservices and cloud computing. During periods of unexpectedly high demand, this inflexibility can cause technical issues for organizations ranging from personal trading platforms to major banks.
