Suites don't rot in the specs — they rot in the harness: five login helpers, three seeding styles, mocks nobody can explain. Agents scaffold fixtures, auth states, network stubs, and matchers brilliantly, but only if the conventions come first and every generated layer stays explainable to the team that inherits it.
Tier: Core · Workflow area: Automation engineering
This playbook uses a coding agent to build the infrastructure layer under your tests — Playwright fixtures, storage-state auth, data seeding, MSW/WireMock stubs, custom matchers — as one coherent, documented system rather than an accretion of ad-hoc helpers. The harness is the highest-leverage and highest-risk place to use generation: everything else depends on it.
When NOT to use:
Write the harness contract. One scratch page, human-authored, stating decisions: auth via storage states per role; data seeded via API (never UI); stubs via MSW with per-test overrides; helpers typed and exported from one place. This document is the review baseline.
Scaffold fixtures from the contract:
Read our conventions in scratch/harness-contract.md and playwright.config.ts.
Create tests/fixtures.ts with typed Playwright fixtures:
- adminPage / memberPage: contexts from role storage states in
.auth/<role>.json, regenerated by a setup project when stale.
- seededOrg: creates an org + users via the API client in
tests/support/api.ts, yields typed handles, deletes on teardown even
if the test fails.
- api: authenticated API client per test.
Comment each fixture with WHY it exists and its cleanup guarantee.
Then write tests/harness.spec.ts that exercises every fixture, and run:
npx playwright test tests/harness.spec.ts
The setup-project auth pattern the agent should produce:
// auth.setup.ts — runs once per role before the suite
setup('authenticate admin', async ({ page }) => {
await page.goto('/login');
await page.getByLabel('Email').fill(process.env.ADMIN_EMAIL!);
await page.getByLabel('Password').fill(process.env.ADMIN_PASSWORD!);
await page.getByRole('button', { name: 'Sign in' }).click();
await expect(page.getByRole('navigation')).toBeVisible();
await page.context().storageState({ path: '.auth/admin.json' });
});
Add network stubbing as data, not logic. Keep handlers thin and responses in fixture files so contract drift is a data diff:
Scaffold MSW for the browser tests: handlers in tests/mocks/handlers.ts
returning JSON from tests/mocks/data/, matching the response schemas in
openapi.yaml. Include a rate-limited (429) and a 500 variant per
endpoint, exported as overrides a test can install for one test only.
Validate every mock file against the OpenAPI response schema in a small
script: npm run mocks:check.
npm run mocks:check && npx playwright test tests/harness.spec.ts
The schema-check script matters: hand-written and generated mocks both drift from the real API, and stubs that lie are worse than no stubs (pb-27 develops this fully).
Generate custom matchers for the assertions your team repeats. toShowToast(text), toBeEmptyState(), toHaveRowCount(n) — each wrapping the waiting and error-message quality that inline assertions skimp on. Require the agent to include a failure-output example in each matcher's doc comment.
Run the explainability review. Harness code gets a stricter review than specs: for every fixture and helper, someone other than the prompter must be able to answer "what does this guarantee, and what happens on teardown failure?" from the code and comments alone. Anything requiring the prompter's memory gets its comments fixed now.
Enforce usage. Add the convention to the pb-08 rules file ("never inline login; use fixtures from tests/fixtures.ts") and a lint/grep gate in CI so the harness stays the only path:
grep -rn "getByLabel('Password')" tests/ --include="*.spec.ts" && exit 1
Cleanup code is where generated harnesses fail silently — a teardown that throws on half-created data leaks state into every later test as unexplainable flake; verify teardown by killing a test halfway (test.fail + early throw) and confirming the next run is clean.
conftest.py fixtures, factory-boy for data, and responses/respx for HTTP stubbing; the explainability review and teardown-kill test transfer unchanged.wiremock/mappings/, generated from the OpenAPI spec and schema-checked in CI; per-test scenarios via the admin API.@org/test-kit) so conventions ship as versioned code to every service; the agent maintains one implementation instead of N drifting copies.mocks:check runs on every PR; a spec change that invalidates a mock file turns the build red.Generate the harness once, well, and reviewed — then make it the only road. A suite where every test rides shared, explainable, schema-checked infrastructure is a suite an agent can safely extend at speed; a suite of private helpers is one no amount of AI will keep honest.