Playwright Test Agents: Planner, Generator and Healer

Since version 1.56, Playwright ships three agent definitions that split test authoring into steps: a planner that explores the app and writes a Markdown test plan, a generator that turns the plan into test files, and a healer that runs failing tests and repairs them. They are not a separate product. They are prompt files plus a dedicated MCP server, installed into your repository and run by the AI client you already use.

This chapter installs them, runs all three on the TodoMVC demo, and reports what each one actually did.

Install the agent definitions#

In a Playwright project (we used @playwright/test 1.65.0):

npx playwright init-agents --loop=claude

--loop selects the AI client. The accepted values in 1.65 are claude, codex, copilot, opencode, vscode and vscode-legacy; npx playwright init-agents --help lists them for your version. For Claude Code the command wrote:

specs/README.md                              directory for test plans
seed.spec.ts                                 default environment seed file
.claude/agents/playwright-test-generator.md  agent definition
.claude/agents/playwright-test-healer.md     agent definition
.claude/agents/playwright-test-planner.md    agent definition
.mcp.json                                    mcp configuration

With --loop=vscode or --loop=copilot the definitions go to .github/agents/*.agent.md and the MCP config to .vscode/mcp.json; the copilot loop also writes a copilot-setup-steps.yml workflow for the GitHub Copilot coding agent. --loop=codex writes TOML files under .codex/agents/. We only ran the agents themselves in Claude Code.

The MCP config does not point at @playwright/mcp. It starts a test-aware server that ships inside @playwright/test:

{
  "mcpServers": {
    "playwright-test": {
      "command": "npx",
      "args": ["playwright", "run-test-mcp-server"]
    }
  }
}

That server has the browser tools from chapter 1 plus tools the agents need: planner_setup_page and planner_save_plan, generator_setup_page, generator_read_log and generator_write_test, and test_list, test_run and test_debug for the healer. Because it uses your playwright.config.ts, the agents get your baseURL, projects and fixtures.

The Playwright docs say to re-run init-agents after upgrading Playwright, so that the definitions match the new tools. Commit the generated files: they are prompts that shape your tests, and they deserve review and a history like any other code.

The seed test#

The agents start every session by running a seed test, which brings the page into a known state. The generated one is an empty test. We moved it to tests/seed.spec.ts and made it open the app:

import { test, expect } from '@playwright/test';

test.describe('Test group', () => {
  test('seed', async ({ page }) => {
    await page.goto('/todomvc/');
    await expect(page.getByPlaceholder('What needs to be done?')).toBeVisible();
  });
});

In a real app the seed is where you log in, reset test data, or open the right tenant. Whatever the seed does, every generated test will do too, so this is the place to put fixtures and setup you want the agents to reuse.

Run the planner#

In Claude Code the definitions are subagents, so you ask for them by name. Our prompt:

Use the playwright-test-planner subagent to create a test plan for the TodoMVC app at https://demo.playwright.dev/todomvc/ using tests/seed.spec.ts as the seed. Limit the plan to 6 scenarios covering adding todos, completing a todo, the Active/Completed filters and clearing completed todos. Save it as specs/todomvc.md.

It wrote a tidy plan with six scenarios, numbered steps and an expect: line under each step, in the format shown in chapter 2. Claude Code reported $0.23 for the run.

Then the summary was honest about something important: the planner had only loaded the seed and looked at the empty page. It never added a todo, clicked a filter or cleared anything. Every expected result in the plan came from the model's knowledge of how TodoMVC apps usually behave. On a well-known demo that works out. On your own application, a plan written that way is a list of plausible guesses formatted as requirements.

The planner's instructions tell it to explore thoroughly; ours did not, in this run. Read the plan before generating anything, and ask the planner to say which expectations it actually observed.

Run the generator#

Use the playwright-test-generator subagent to generate tests for scenarios 1.1, 1.3, 1.4 and 1.6 of specs/todomvc.md (one subagent call per scenario, in sequence). Seed: tests/seed.spec.ts.

The generator works step by step: it runs the seed through generator_setup_page, performs each step in the live browser, reads back the log of Playwright calls it made, and writes one file per scenario with generator_write_test. Each step's text from the plan becomes a comment above the code for that step, which makes review much easier.

Four files, 172 seconds of model time, $0.40. In its summary it listed what it had not confirmed: that the selected filter gets a selected class, and the final empty-state checks, because its last click had hit a stale element reference. That list was the most useful part of the output. Three tests passed on the first run; the fourth had the strict mode violation dissected in chapter 2.

Run the healer#

Use the playwright-test-healer subagent to fix the failing tests in tests/todomvc.

The healer runs the suite with test_run, opens each failure with test_debug, inspects the page, edits the test and runs it again until it passes. It changed one line (the .all() loop from chapter 2), the suite went to 4 passed, and the run cost $0.16.

Read the healer's own instructions before you rely on it. Two lines from the definition that init-agents writes:

- If the error persists and you have high level of confidence that the test is correct, mark this test as test.fixme()
- Do not ask user questions, you are not interactive tool, do the most reasonable thing possible to pass the test.

"Do the most reasonable thing possible to pass the test" is a fine instruction for a broken locator and a dangerous one for a broken application. Chapter 4 is about that difference.

The test that encoded a bug#

To see how the generator handles a wrong application, we wrote a one-page shop: a 12.50 EUR bag of coffee, a quantity field, an "Add to bag" button, and a banner reading "Free shipping on orders over 20 EUR". The page has a bug: it charges 4.95 EUR shipping on every order. The plan said only:

1. Set the quantity to 2 and add the product to the cart.
  - expect: The cart summary shows the quantity, the total and the shipping cost.

The generator produced this, and it passes:

await page.getByRole('spinbutton', { name: 'Quantity' }).fill('2');
await page.getByRole('button', { name: 'Add to bag' }).click();

// expect: The cart summary shows the quantity, the total and the shipping cost.
await expect(page.getByText('2 x House blend, total 25.00 EUR, shipping 4.95 EUR')).toBeVisible();

Its chat summary even said: "the page says 'Free shipping on orders over 20 EUR', but a 25.00 EUR total was charged 4.95 EUR shipping. The test asserts the 4.95 EUR as observed." The warning went into a chat transcript that nobody keeps. The bug went into the test suite, where it protects the wrong behaviour from being fixed.

When the plan stated the correct value (in a separate run against a version where the total ignored the quantity, the plan said "the correct total"), the generator asserted 25.00 EUR and the test failed, which is what you want. The difference was not the model. It was whether the plan carried the oracle.

What it cost, and what that buys#

Our full loop (plan, four tests, one heal) came to about $0.80 at list price and a few minutes of model time, plus our review. That is cheap compared with writing four tests by hand. It is not free, and it repeats every time someone regenerates instead of editing. Chapter 7 covers budgets.

The agents are best at the mechanical work: finding locators, writing the steps, fixing a renamed button. The planner's expectations and every assertion still need a person who knows what the application is meant to do.