Test data is where QA meets the law: a production extract pasted into a public chatbot is a compliance incident, not a shortcut. This module builds the privacy-safe pipeline — PII masking before any prompt, schema-constrained synthetic data, adversarial datasets — then applies calibrated trust to accessibility, i18n, performance and compatibility testing with AI.
Two forces make this module first-class rather than an appendix. India's DPDP Act 2023 and its 2025 Rules put real penalties — up to ₹250 crore — behind personal-data mishandling, and GDPR has enforced the same discipline in Europe for years; production-derived test data is squarely in scope. Meanwhile the services reality is that testers on client accounts often may not paste client data into public AI tools at all, per contract. The engineers who can generate realistic, schema-valid, legally clean test data — and who know exactly what may enter which tool — are solving a problem their whole account has.
Meera — senior QA on a payments integration project at a services company, client is a mid-size lender — needs realistic loan-application data for edge-case testing. The staging database is stale. A teammate offers the pragmatic shortcut everyone has seen: "grab a production extract, scrub the obvious columns."
The extract has 12,000 rows. Names and PANs are in the obvious columns — but PII is also inside free-text fields: phone numbers in agent notes, an Aadhaar number typed into a comments field, email addresses in dispute descriptions. Column-level scrubbing misses all of it.
A junior on the team, trying to be helpful, pastes 50 raw rows into a public chatbot to "generate more like these." Meera catches it in review. Client personal data has now left the approved boundary — a reportable event under the client contract, and DPDP exposure for real.
The team builds the pipeline this module teaches: Microsoft Presidio scans every field — free text included — and masks findings; anything sensitive that needs LLM processing runs on a local model via Ollama; synthetic data is generated to the Pydantic schema and validated in a loop; and a one-page decision tree says what may enter which tool, signed by the account lead.
The synthetic dataset is better than the extract: referential integrity intact, distributions realistic, plus the edge cases production rarely coughs up on demand — Unicode names, leap-day birthdates, boundary-length address fields. And nothing in it maps to a real human.
The near-miss is the norm, not the outlier: column-level thinking about PII, helpful juniors, and public tools one paste away. The fix is not vigilance — vigilance doesn't scale — it is a pipeline where the safe path is also the easy path.
The pipeline has a strict ordering, and the ordering is the compliance argument: nothing derived from production touches an external model until PII detection and masking have run.
The decision tree — the artifact auditors and account leads actually want — fits in one table:
| Data class | Public AI tools | Enterprise/approved endpoint | Local model | Notes |
|---|---|---|---|---|
| Raw production data | Never | Never | Only inside the client boundary, if contract allows | Presidio scan first, always |
| Masked/pseudonymized extract | No | With client sign-off | Yes | Masking must be verified, not assumed |
| Fully synthetic data | Yes | Yes | Yes | Provenance documented — "synthetic" must be provable |
| Client requirements & defect text | Per client contract | Usually yes | Yes | Free text carries PII too — scan it |
PII lives in free-text fields — agent notes, comments, descriptions — not just in labeled columns. A masking step that only touches the schema-obvious columns produces an extract that looks clean and isn't, which is more dangerous than raw data because people stop being careful with it.
Realistic data proves the happy path; adversarial data finds the bugs. Generate it by category with risk tags, not as a grab bag — coverage claims need structure:
Environment dependence, meanwhile, is a top flake cause from Module 4 — and mocks are the fix. WireMock/MSW/Prism fakes generated from OpenAPI specs or recorded traffic give you deterministic dependencies, and the mock must track the contract: a mock that drifts from the real API converts integration bugs into "passing" tests. Regenerate from the current spec on a schedule, and keep one real-integration path in the pipeline as the drift alarm. The same codegen discipline scaffolds docker-compose test stacks and CI workflow files — with every line explained, because an environment nobody understands is an outage nobody can fix. If the agent writes your compose file, the agent also writes the line-by-line commentary, and you read it (UC-30, UC-31).
Accessibility testing is where this track's central skill — knowing exactly how much to trust an automated layer — shows up in its purest form, because the numbers are published and stark: automated scanners catch only roughly 30–40% of WCAG issues. Practitioner consensus, not vendor pessimism. The protocol that respects those numbers has three layers:
axe-core or Pa11y scans every page state — cheap, fast, in CI. Then the LLM turns each violation into something actionable: the WCAG criterion cited, why it matters to a real user, a suggested fix in your framework's idiom. This is where AI genuinely helps: a scan report developers understand gets fixed; a wall of rule IDs gets ignored.
Keyboard-only navigation: every interaction reachable, focus visible, no traps. Screen-reader walkthrough: does the experience make sense — reading order, announcements, context? Scanners verify attributes exist; they cannot verify the page is usable. A logical-seeming DOM can be an incomprehensible listening experience, and only a human at the controls finds that out.
The deliverable format is the discipline: an a11y report that explicitly separates scanner-caught from human-caught findings. It keeps the team honest about what automation buys, builds the case for manual testing time with evidence, and — in India — maps to rising enforcement of accessibility requirements for government and financial services work.
"We run axe in CI, so we're covered" is the accessibility equivalent of "the suite is green." You are covered for the ~30–40% of issue classes scanners can see. The remaining majority is keyboard traps, focus chaos and reading-order nonsense — findable only by hands and ears.
Three specialist lanes, one recurring pattern — AI shortens the mechanical ends, humans own the judgment in the middle:
Mask before you prompt, validate what you generate, and calibrate trust per layer: scanners catch a minority of a11y issues, mocks drift unless re-anchored, and perf hypotheses are guesses until APM confirms them. The pipeline where the safe path is the easy path — Presidio, local models for the sensitive step, schema-validated synthesis — is what keeps test data useful and legal at once.
Take this into Lab 08 (privacy-safe test data factory) — the fake production extract has an answer key of planted identifiers — and Lab 10 (accessibility with calibrated trust), where finding the seeded scanner-invisible bugs is the pass condition. Companion playbooks: pb-24 through pb-33.