Operations | Monitoring | ITSM | DevOps | Cloud

This blog post is part of PagerDuty’s ongoing series on how we’re helping customers navigate their journey towards autonomous operations. Read on to learn about how PagerDuty’s AI-Powered Incident Communications builds towards this vision.

Today we're launching Foresight AI, Gremlin’s agentic resilience product that analyzes and tests your systems for potential failures, fixes them, and verifies reliability at the speed of AI. I've spent most of my career on call. At Amazon and Netflix, I served as a Call Leader, the person running the bridge when something big broke. Those years taught me the same lesson we founded Gremlin on: the best incident is the one that never happens.

Application development and operations teams are adopting AI coding tools at an exponentially increasing rate, from a rounding error of 6% of code output by AI in 2023, to as much as 51%-75% for a majority of enterprises. Agents are making pull requests (PRs) faster than human developers could have ever dreamed. As features are pushed to market faster, there’s a sharp increase in production incidents, with 80% of development shops specifically tracing production outages to AI.

Dedicated vector databases have become such a standard part of the RAG conversation that teams often add one before they have proved they need it. According to studies, over 70% of companies using LLMs are using vector databases and RAG to customize their models. That shows how quickly the pattern has become normal. However, it does not mean every RAG application needs a separate retrieval system. If your application already runs on PostgreSQL, pgvector may be enough.

Why is the telemetry AI agents generate so intimidating? If you turn on Claude Code’s internal telemetry it’ll throw a wall of text at you. And, it’s very expensive to store. But, the bigger issue is that you can’t make sense of it. Luckily it’s all OpenTelemetry native. That means you can configure it to send, transform, and store what you really need. Which raises the only question that matters. What do you actually need?

Most AI spend comes in with no tags and no owner attached. Your provider console shows total spend, maybe broken out by API key or model. It won’t tell you that the sales team spent $1,700 on Claude this week, let alone what the work was. And the problem is growing. McKinsey found that 56% of organizations now use AI in three or more business functions. More teams means more spend, and most companies respond with a spending cap. Set it too low and you slow down the work you wanted AI to help with.