Automated scanners catch roughly a third of WCAG failures, AI can explain and fix what scanners find, and neither can tell you whether a screen-reader user can actually complete checkout. This playbook layers all three — scanner, AI, human — with explicit trust boundaries between them.
Tier: Core · Workflow area: Non-functional & visual testing
The output is a repeatable three-layer protocol: axe-core in CI as the regression floor, an AI pass that turns raw violations into cited, prioritized fixes, and a scripted manual pass for the keyboard and screen-reader coverage automation cannot replace.
@axe-core/playwright) or Pa11y for URL sweeps.Layer 1 — put the scanner in CI as a floor, not a goal.
const results = await new AxeBuilder({ page })
.withTags(['wcag2a', 'wcag2aa', 'wcag21aa'])
.analyze();
expect(results.violations).toEqual([]);
npx playwright test a11y.spec.ts --reporter=line
1) checkout page: 4 violations
color-contrast (serious) ×3 · label (critical) ×1
Gate on new violations only at first (snapshot the current count per page); a hard zero-gate on a legacy app gets the job deleted within a week.
Layer 2 — AI turns violations into cited fixes. Feed the assistant the JSON plus the offending component source:
Here is an axe-core violations array for our checkout page, and the
CardForm.tsx source. For each violation:
1. Explain the failure in one sentence a developer will not argue with.
2. Cite the exact WCAG success criterion (number + name + level).
3. Propose the minimal code fix — no redesigns.
4. Say who is affected and how (screen reader, low vision, motor).
Rank by user impact, not by axe's severity field.
Do not invent criteria numbers; if unsure, say so.
Sample of what good output looks like:
1. The card-number input has no programmatic label (placeholder ≠ label).
WCAG 1.3.1 Info and Relationships (A) / 4.1.2 Name, Role, Value (A).
Fix: <label htmlFor="cardNo"> wired to the input; keep placeholder
as example text only. Affects: screen-reader users hear "edit text,
blank" and cannot tell which field this is.
Verify the citations — this is the calibration step. Spot-check every WCAG number against the actual spec (or your bookmarked quick-ref) before the report leaves QA. Models occasionally cite a plausible-but-wrong criterion; one bad citation and developers stop trusting the whole report. Trust the explanation and fix draft; verify the citation and the severity.
Layer 3 — the scripted manual pass. For each critical journey (login, search, checkout), run a fixed script no tool can automate:
Ask the AI to generate this checklist per screen from the component tree — it is good at enumerating interactive elements to test — but a human executes it. Time cost: 20–30 minutes per journey, and it finds the failures that lose lawsuits.
File findings with the layer attached. Every ticket states its evidence class: scanner (deterministic, non-negotiable), ai-explained (verified citation), or manual (human judgment). This is what "calibrated trust" means operationally — the reader always knows how much to trust the finding and how it was produced.
Never let an AI mark an a11y issue as a false positive on its own. Downgrading a finding is a human decision — models are eager to agree that an issue "may not affect real users," and that sentence has no place in a QA report without a human behind it.
pa11y-ci --sitemap https://app.example/sitemap.xml for breadth; keep axe-in-Playwright for authenticated flows.Calibrated trust means each layer does what it is actually good at: the scanner regresses, the AI explains and drafts, the human judges. Collapse the layers in either direction — trusting AI to judge, or humans to scan at scale — and the protocol fails.