Lab 10 · Accessibility with Calibrated Trust

Scanners catch roughly 30–40% of accessibility issues; the industry keeps re-learning this number and the vendors keep letting people forget it. Deepa's team runs axe in CI, sees zero violations, and calls the release accessible — which is exactly the calibration failure this lab attacks. The page below plants both kinds of defect: machine-visible ones (axe will flag them, and AI will explain them well) and machine-invisible ones — technically conformant markup that is unusable by keyboard or absurd through a screen reader. Your artifact is an a11y report with two honest columns: what the scanner caught, and what only a human could.

Time, tier & cost

~75 minutes hands-on · Core tier (maps to module M5 and playbook PB-29). Cost: $0 (axe-core CLI, Playwright, a free AI tier; screen reader = VoiceOver on macOS or NVDA on Windows).

Setup

mkdir -p qa-lab-10 && cd qa-lab-10 && npm init -y
npm install --save-dev @axe-core/cli @playwright/test axe-playwright serve
npx playwright install chromium

index.html — the page under test. Author it exactly; every oddity is load-bearing:

<h1 style="color:#9a9a9a;background:#ffffff">Team Settings</h1>
<img src="avatar.png" alt="image">
<p>Manage who can access your workspace below.</p>
<label>Search members <input type="search" id="q"></label>
<input type="text" placeholder="Invite by email">
<div role="button" tabindex="0" id="invite-btn"
     style="outline:none;border:1px solid #888;padding:6px;width:120px">
  Send invite
</div>
<h2>Members</h2>
<div id="menu-wrap">
  <button id="role-menu-btn" aria-haspopup="true" aria-expanded="false">
    Change role
  </button>
  <div id="role-menu" hidden>
    <div class="opt" tabindex="0">Viewer</div>
    <div class="opt" tabindex="0">Editor</div>
    <div class="opt" tabindex="0">Admin</div>
  </div>
</div>
<p><span style="color:#c0392b">Red rows</span> indicate suspended accounts.</p>
<table>
  <tr><td>Asha Rao</td><td>Editor</td></tr>
  <tr style="color:#c0392b"><td>Vikram Nair</td><td>Viewer</td></tr>
</table>
<a href="#" onclick="location='help.html'">Click here</a> for role documentation.
<script>
  const btn = document.getElementById("invite-btn");
  btn.addEventListener("click", () => alert("Invite sent"));
  const menuBtn = document.getElementById("role-menu-btn");
  const menu = document.getElementById("role-menu");
  menuBtn.addEventListener("click", () => {
    menu.hidden = false;
    menuBtn.setAttribute("aria-expanded", "true");
    document.querySelectorAll(".opt").forEach((o) => {
      o.addEventListener("keydown", (e) => {
        if (e.key === "Tab") e.preventDefault(); // keep focus in menu
      });
    });
  });
</script>

Serve it: npx serve . (note the port). Create a11y-report.md with two headings: "Scanner-caught" and "Human-caught."

Steps

1. 1Run the scanner and freeze its output.npx axe http://localhost:3000 --save axe-results.json (adjust port). List every violation in the "Scanner-caught" section: rule id, impacted node, severity. Expect a handful — contrast, image-alt quality is partially checkable, form labeling. Do not fix anything yet.
2. 2Have AI explain each violation like a teacher. Paste the axe JSON into Claude/ChatGPT: "For each violation: the WCAG 2.1 success criterion number and name, who this harms in practice, and the minimal fix. Cite the criterion for every claim." Verify two of its WCAG citations yourself against the actual criterion text (models occasionally cite a plausible-but-wrong SC number — log it if yours does). Append the explanations to the report.
3. 3Predict before you touch. Write down, before the manual pass: "What will the scanner have missed?" List candidate classes: keyboard operability, focus visibility, focus traps, reading order, meaningful alt quality, color-as-only-signal, link purpose. This prediction is your calibration snapshot.
4. 4Keyboard pass — no mouse for ten minutes. Tab through the entire page and attempt every task: search, type an email, send an invite, open the role menu, pick a role, follow the docs link. Log every failure in "Human-caught" with the exact interaction that broke. At minimum you should hit: an element you can reach but not activate with Enter/Space, an element with no visible focus anywhere, and a menu you can enter but never leave.
5. 5Screen-reader pass — fifteen minutes. VoiceOver (Cmd+F5) or NVDA. Navigate by headings, then straight through. Log what each seeded oddity sounds like: the avatar image, the invite field, the "Red rows" convention, the "Click here" link, the table. The scored question per item: does the audio convey what a sighted user gets, and would you know what to do next hearing only that?
6. 6Sort every finding into the two columns. For each human-caught finding, note why the scanner missed it — usually because the markup is technically legal (role="button" with tabindex satisfies the rule; the rule cannot know no key handler exists) — and tag it with its WCAG SC (AI may help; verify citations).
7. 7Fix and re-verify both classes. Fix everything: real alt text; visible focus styles; a <label> for the email input; Enter/Space activation on the invite control (or make it a real <button> — the better fix, say why); Escape/Tab release plus arrow-key support for the menu; a non-color indicator for suspended rows; a descriptive link. Rerun axe (clean) and redo the keyboard pass (every task completable). The double re-verify — scanner and hands — mirrors the two-column report.
8. 8Write the calibration paragraph. End the report with the numbers: X findings scanner-caught, Y human-caught, and your predicted-vs-actual from step 3. Then one sentence you'd put in a team playbook about what a clean axe run does and does not mean.

Verify

• •
  axe-results.json saved; every violation explained with a verified WCAG citation.
• •
  Prediction list written before the manual pass.
• •
  Keyboard pass log shows the unreachable-activation, invisible-focus, and trapped-menu findings with exact interactions.
• •
  Screen-reader log covers the image, the unlabeled input, color-only signaling, and the link text.
• •
  Post-fix: axe clean and all tasks keyboard-completable; both re-verifications recorded.
• •
  Final report has the two columns and the calibration paragraph with counts.

Self-grading rubric (pass = 7/10):