Operations | Monitoring | ITSM | DevOps | Cloud

Can you actually trust an AI agent? In this pre-recorded episode of The Context Window, Nicole van der Hoeven sits down with Yas Ekinci, an engineer on the Grafana AI team, to talk about evals — how Grafana measures the quality and reliability of the AI it ships. They get into the difference between online and offline evals, why reviewing AI-generated code has become the real bottleneck, the "final answer problem" of plausible-but-wrong outputs, and o11y-bench, Grafana's open benchmark for observability agents. Along the way.

If your incident response still depends on engineers stitching alerts together… You don’t have observability. You just have dashboards. Correlation has to be automatic, contextual, and system-aware, or it becomes expensive noise.

"CFEngine: The agent is in" is our monthly webinar series, where we show new features, teach best practices, and keep the community informed about everything CFEngine.

At LLMday Lisbon, our Software Engineer, Viktor Vasylkovskyi, highlights the realities of building production AI agents with LangGraph - sometimes getting it right, often learning the hard way. This talk is about what was actually shipped, including a distributed multi-agent setup at PagerDuty. Viktor breaks down the real tradeoffs between LLM-driven and deterministic orchestration, what broke, and how he’d approach it differently now.