The Checkly Playwright Reporter: Live Demo, Rocky AI RCA & Production Monitoring

Your Playwright tests catch bugs. The hard part is figuring out what actually broke — and sharing that context with your team. This session shows exactly how the Checkly Playwright Reporter solves that: one shared home for all your test runs, AI-powered root cause analysis, and a direct path from failing test to production monitor.

María de Antón, PM for Playwright features at Checkly, runs a live demo on a real app with real failures.

0:00 Welcome and introductions

1:00 What we're covering today

1:45 Checkly overview — the active reliability layer for developers and agents

2:30 Detect, communicate, resolve — and Rocky AI

3:15 Why we built the Playwright Reporter: the three gaps

4:10 The "AI bolt-on" problem: why pasting traces into ChatGPT isn't enough

5:00 The full picture: same Playwright code, from CI test to production monitor

6:00 Live demo starts — meet Raccoon Records

7:00 The demo repo: publicly hosted on GitHub, open to follow along

7:45 Configuring the Checkly reporter in playwright.config.ts

8:15 Secret scrubbing, custom session names, and verbose mode

9:00 Running 123 Playwright tests with the reporter live

10:00 Setting up your Checkly API key and account ID (trial account walkthrough)

11:00 Test sessions in Checkly — failures grouped by project, errors surfaced first

12:00 Rocky AI root cause analysis — impact, cause, and suggested fix from the trace

13:15 Adding extra context to Rocky + OpenTelemetry backend traces

14:00 Network logs, browser console, screenshots, and videos per test run

14:45 Test suite timeline — CPU and memory usage across parallel test runs

15:00 Error groups — frequency, history, and root cause across all runs

16:15 Filtering test sessions by author and custom tags

17:00 Checkly config as code — declaring checks from Playwright tags

17:45 npx checkly test --record — and why --record is becoming the default in CLI v8

18:30 Promoting tests to production monitors with npx checkly deploy

19:15 Running checks across global locations (us-east-1, eu-west, and more)

19:45 Flaky test detection — degraded status before it becomes a failure

20:15 Analyzing a 500 error response with Rocky AI

21:00 Source code view inside a check run — Playwright config and test code in context

22:00 Error group resolution and code fix suggestions

23:00 Deploying to production and watching monitors go live

24:00 CLI check status and the incident workflow (npx checkly incident)

25:00 Test suite analytics — flakiness trends, duration spikes, query builder

26:00 Q&A: installing the reporter and next steps

26:30 Q: Does the reporter work with CircleCI or GitHub Actions?

27:30 Q: How is the reporter different from Checkly checks?

28:30 Q: How do you visualize results per app or team?

29:30 Q: Why Checkly if Claude Code is already running your Playwright tests?

32:00 Closing — where to find us

Install the reporter (free on all plans): https://www.checklyhq.com/docs/detect/testing/playwright-reporter/
Reporter changelog: https://www.checklyhq.com/docs/detect/testing/playwright-reporter-changelog/
NPM package: https://www.npmjs.com/package/@checkly/playwright-reporter
Demo repo (Raccoon Records): https://github.com/checkly/playwright-reporter-demo
Start a free trial: https://app.checklyhq.com/signup
Join the Checkly Slack community: https://www.checklyhq.com/slack

#Playwright #TestAutomation #Checkly #QA #DevOps #CI #EndToEndTesting #PlaywrightTesting #AI #SoftwareTesting