Operations | Monitoring | ITSM | DevOps | Cloud

Your agent breaks in production. You fix it and move on. But the input that actually broke it is gone and two months later the same failure quietly comes back, because there was never anything to test against. That's not a debugging problem. It's a missing dataset. This demo turns low-scoring production traces into a regression suite you can run against every prompt and model change, without writing a single test case by hand.

Every engineering team has unit tests. They tell you the code still works. They tell you nothing about what the model started saying. This demo wires a single eval gate script into a git pre-push hook, so Splunk Agent Observability scores every agent's output before the push is allowed through. Luna, an on-premise small language model, runs as a synchronous judge against fixed thresholds. Fail one, and the push is blocked.

Most teams keep an agent from regressing by hardcoding checks into its logic, an if-statement here, a regex there. Every new rule then becomes a code change, a review, and a deploy, and the person who spots the problem in production is rarely the person who can ship the fix. Agent Control moves those rules out of the code and into one hub. Steer, block, and validate agent behavior in real time, with rules any team member can update without touching the codebase.

One of the most expensive failure modes in AI agents is a silent retry loop. An agent hits a rate limit, blindly retries until it succeeds, the user gets a response, and no ticket ever gets filed. Under the hood, tokens are burning, latency is compounding, and a transient error today becomes a sustained outage tomorrow.

Your AI agent just approved a return it shouldn't have, and nothing threw an error. Here's one agent request as a trace: the agent span, LLM calls, tool calls, and retrieval, with tokens and latency at every step. The fix wasn't the prompt. It was the data.

The future of operations is predictive, unified, and AI-powered. Dive into how Splunk and Cisco help organizations unlock cross-domain insights at massive scale, provide intelligent context across security and observability, and build trust in AI with end-to-end visibility. Hear directly from product leaders on the latest innovations shaping agentic enterprises.

Join Jeetu Patel, Head of Product at Cisco, and Kamal Hathi, GM at Splunk, alongside a special guest, to see how organizations transform security, observability, and operations with AI workflows that are proven at scale. Discover how Splunk and Cisco bring unified visibility, trusted AI, and intelligent automation to help organizations.

Close the AIOps loop with Splunk Observability Cloud and Red Hat Ansible Automation Platform. Explore how OpenTelemetry and Event-Driven Ansible turn alerts into automated fixes, reducing manual firefighting and speeding up incident resolution.

Most teams keep an agent from regressing by hardcoding checks into its logic, an if-statement here, a regex there. Every new rule then becomes a code change, a review, and a deploy, and the person who spots the problem in production is rarely the person who can ship the fix. Agent Control moves those rules out of the code and into one hub. Steer, block, and validate agent behavior in real time, with rules any team member can update without touching the codebase.

Your agent breaks in production. You fix it and move on. But the input that actually broke it is gone and two months later the same failure quietly comes back, because there was never anything to test against. That's not a debugging problem. It's a missing dataset. This demo turns low-scoring production traces into a regression suite you can run against every prompt and model change, without writing a single test case by hand.