Author E2E Tests via Playwright MCP with ARIA Snapshots

Tier: Core · Workflow area: Testing & quality

Hand-written E2E suites lag the app by months because writing them is tedious. Agent-authored suites arrive the same day — and arrive with agent habits: discovered-state assumptions, brittle waits, and assertions on whatever happened to be on screen. This playbook keeps the speed and reviews out the habits.

Prerequisites

• •
  Playwright installed in the repo; Playwright MCP (@playwright/mcp) wired into your agent (Claude Code, or Copilot's MCP support).
• •
  A test environment with deterministic seed data and test accounts — the agent will need credentials you are comfortable putting in an env var, never in the spec.
• •
  A playwright.config.ts with sensible defaults (baseURL, trace on retry) so generated specs inherit them.

The recipe

1. 1.

   Have the agent walk the journey before writing anything. Exploration first, code second:

   Copy

   Using the Playwright MCP browser against $BASE_URL (test env, login
   with $TEST_USER/$TEST_PASS from env): complete this journey — search
   for a product, open its detail page, add to cart, check out with the
   test card, reach order confirmation. Narrate each step: what you
   interacted with, identified by role and accessible name. If you get
   stuck or must guess, say so — do not improvise through modals or
   cookie banners silently. Do not write the spec yet.
   

   The narration is your first review artifact. "Clicked the button with role button, name 'Add to cart'" is a good sign; "clicked the third div in the results grid" predicts the selector quality you are about to get, and flags a missing accessible name in the app itself.

2. 2.

   Then request the spec with the determinism rules stated.

   Copy

   Now write tests/checkout.spec.ts for that journey. Rules:
   - Locators: getByRole/getByLabel/getByText with accessible names;
     data-testid only where no accessible name exists (list each such
     case at the top of the file as a TODO to fix the markup).
   - No waitForTimeout, no networkidle waits. Rely on web-first
     assertions (expect(...).toBeVisible/toHaveText) and auto-waiting.
   - Assert structure at each key stage with toMatchAriaSnapshot —
     scoped to the relevant region, not the whole page.
   - Test data: reference seeded fixtures by name; no assertions on
     values that vary per run (dates, order numbers) except via patterns.
   - Each test independent: fresh context, no ordering dependencies.
   

3. 3.

   Scope the ARIA snapshots or they become the new brittleness. A full-page snapshot fails when the footer changes. The generated assertions should look like:

   Copy

   await expect(page.getByRole('region', { name: 'Order summary' }))
     .toMatchAriaSnapshot(`
       - region "Order summary":
         - heading "Order summary" [level=2]
         - list:
           - listitem: /Aeron Chair/
         - button "Place order"
     `);
   

   Region-scoped, roles and names rather than text dumps, regex for variable content. Anything asserting the entire page tree, exact prices, or timestamps gets rewritten. ARIA snapshots earn their keep precisely because they check what assistive tech perceives — a div-soup regression that keeps pixels identical still fails the snapshot.

4. 4.

   Run the determinism review — the step that decides whether this playbook worked. Read the spec for the agent tells: discovered-state coupling (the test assumes the cart already had an item because it did during exploration — must start from seeded, stated preconditions); overfit assertions (asserting incidental content that happened to render); hidden ordering (test B passes only after test A ran); swallowed instability (a try/catch or conditional click around a flaky dialog — that is a bug to surface, not code to keep). Fix in conversation with the agent; it revises fast when told exactly which rule a line violates.

5. 5.

   Prove stability before merging: ten consecutive runs.

   Copy

   npx playwright test checkout.spec.ts --repeat-each=10 --workers=4
   

   40 green runs (across parallel workers, which also flushes out shared-state bugs) is the admission ticket to CI. Any flake now is infinitely cheaper than the same flake at 2 a.m. in a release pipeline — diagnose it with the trace viewer, fix the root cause, and re-run the gauntlet.

6. 6.

   Keep authorship cheap forever. Commit the exploration-plus-rules prompts into the repo (a tests/AUTHORING.md or your agent instruction file). New journey, same recipe: explore, generate, review, gauntlet. The suite now grows at the speed of the app instead of trailing it.

Tool variations

• Claude Code + Playwright MCP: the reference setup — exploration, generation, and the repeat-each gauntlet all in one session, with you reviewing the spec diff.
• Copilot agent mode with MCP: same flow inside VS Code; keep the determinism rules in the instruction file so every generated spec inherits them.
• Playwright codegen as fallback: no MCP available? npx playwright codegen records your own walkthrough, then have any assistant refactor the recording to the same rules — less autonomous, same review discipline.
• Mobile web / responsive: rerun the journey under devices['Pixel 7'] projects; mobile layouts change roles and visibility, so snapshots are per-project, not shared.

Quality check

• Exploration narrated by role and nameand every "no accessible name" moment converted into an app-markup TODO, not a testid workaround.
• Zero fixed waitsno waitForTimeout or networkidle; web-first assertions throughout.
• Snapshots scopedregion-level ARIA snapshots with patterns for variable content; no whole-page trees.
• Preconditions explicitevery test states and seeds its own starting state; runs green in isolation and in parallel.
• Gauntlet passedten consecutive clean repeats on record in the PR before the spec enters CI.