Operations | Monitoring | ITSM | DevOps | Cloud

On 28 September, Anthropic published a playbook for designing evals and hillclimbing against them, and shipped two commands that put it to work. In Claude Code, /claude-api build-eval interviews you and builds an eval inside your codebase. /claude-api hillclimb then improves your app against it, one change per round, with a held-out split to catch overfitting. It is a good piece of work, and the results are concrete.

Ask a team running an agent in production whether last week’s change made it better. Most can’t say. They swapped a model or rewrote a skill, eyeballed a few runs, and moved on. That works for a demo. It doesn’t work for an agent that touches production, where customers want two things that pull against each other: precision where a mistake is expensive, and low cost where the work is routine. A root cause someone acts on has to be right.

An agent that works on your laptop is borrowing three things it doesn’t own: your identity, your attention, and your judgment. It runs with your credentials and your access. It runs when you trigger it. And when it gets something wrong, you notice and tell it, which is the only evaluation it ever gets. Deploy the same skill to the cloud, unattended and pointed at production, and all three disappear at once.

AI coding tools have changed how fast software ships. They haven’t changed who runs it. Every extra release, alert and incident still lands on the same SRE, Platform and DevOps teams, and 78% of technology leaders report more incidents since adopting AI-generated code. So operations teams are turning to agents too. Six in 10 enterprise leaders are piloting, deploying or evaluating automated incident response, according to Caylent.

In July 2025, nine days into a public build experiment, Replit’s coding agent deleted a user’s production database. The user had declared a code freeze and told the agent not to make changes without his approval. The agent ran destructive commands anyway, wiping records on more than 1,200 executives and over a thousand companies, and then wrongly reported that the data couldn’t be rolled back. The instruction was clear, and the agent had it in context.

Your agent has earned trust in shadow mode. Now it runs on its own: an alert fires, the agent starts, investigates and proposes a fix before anyone opens a laptop. Here is what it takes to make that safe, scalable and better every week. This is the third article in a series on taking an AI SRE agent from a weekend experiment to production. Part 1 built a local, read-only agent on a throwaway cluster and refined it against a synthetic eval set.

Picture a Monday at 03:10 UTC. An agent investigating an out-of-memory alert on payment-service works out the pattern: it runs out of memory every Monday between 03:00 and 04:00, and the spike lines up with the batch reconciliation job. The next Monday the alert fires again. A different agent picks it up and starts from zero, because nothing it can see holds what the first one learned.

Komodor spent years building an AI SRE platform before the category had a name. With the launch of the Komodor Agentic Operations Platform, it’s opening that engine up so enterprises can build, run, govern and optimize their own agents in production. Following the launch, co-founder and CEO Ben Ofiri sat down to talk about why now is the right time for agentic operations, what breaks between prototype and production, and where operations will head next.