Generate UAT Test Cases That Expose Their Own Coverage Gaps

Workflow area: UAT & Testing · Tier: Core · Hands-on time: ~35 minutes

UAT packs written by hand converge on the happy path, because writing test case forty-one is tedious and the deadline is real. AI removes the tedium — but an AI-generated pack has its own signature failure: plausible tests that quietly skip the requirement nobody phrased crisply, plus tests that verify behavior no requirement asked for. The coverage matrix is the instrument that catches both, which is why in this playbook the matrix is not documentation of the pack — it is the quality gate the pack must pass.

When to use — and when not to

• Use it when stories and acceptance criteria are baselined and UAT planning startsthe generation-plus-matrix pass turns a week of drafting into a day of reviewing.
• Use it mid-UAT to cluster incoming defectsmapping defects onto requirements and test cases shows whether failures concentrate in one requirement area or one test design flaw.
• Use it on regression scope for change requestsregenerate the matrix after a CR to see which tests the change invalidates.
• Skip it when acceptance criteria are untestable"the system shall be user-friendly" generates garbage tests; run the criteria through a testability pass first and come back.
• Skip business-priority judgmentswhich requirements are critical-path, which failures block go-live: that weighting comes from the business owner, not the generator.
• Skip production data in test casesevery generated case that needs data gets synthetic data; if a draft test says "use customer 44812," that is a defect in the test.

The recipe

1. 1.

   Assemble the tested set. Collect the baselined stories with IDs, their acceptance criteria, and any relevant NFRs slated for business verification. Note which requirements the business owner has flagged critical-path — the matrix will weight review effort by this.

2. 2.

   Generate cases and matrix together. One prompt, both artifacts, so they cannot drift:

   Copy

   You are drafting a UAT pack for business testers. Attached:
   stories with IDs and Given/When/Then acceptance criteria.
   
   Produce:
   1. TEST CASES — for each acceptance criterion at least one
      case: TC-ID | traces-to (story + criterion ID) | title |
      preconditions | numbered steps in business-user language
      (what to click/enter/observe — no API calls) | test data
      (synthetic only, realistic formats) | expected result
      (observable, unambiguous) | priority inherited from the
      requirement.
      Include negative cases (invalid input, missing data,
      unauthorized user) and boundary cases wherever a
      criterion implies a limit.
   2. COVERAGE MATRIX — rows: every story and criterion ID.
      Columns: covering TC-IDs, case count, coverage verdict
      (COVERED / PARTIAL: <what is untested> / NONE).
   3. ORPHANS — every test case that traces to no criterion.
      Do not delete them; list them for adjudication.
   Rules: never invent requirements to justify a test. If a
   criterion is too vague to test, output UNTESTABLE with the
   reason instead of a weak test.
   

3. 3.

   Read the matrix before the tests. The verdict lines are the findings:

   Copy

   ST-07/AC-2 (partial refunds) | — | 0 | NONE
   ST-11/AC-1 (duplicate submission) | TC-31 | 1 | PARTIAL:
     tested via double-click only; concurrent-session path
     untested
   ORPHAN: TC-44 "verify export includes archived records" —
     no requirement mentions archived records in export
   UNTESTABLE: ST-09/AC-3 "statement generation is fast"
   

   Each NONE is either a generation miss (regenerate with the criterion quoted) or a requirement so vague it repelled testing (fix the requirement). Each orphan is either scope creep to delete or — more interestingly — evidence of a missing requirement to raise. Each UNTESTABLE goes back through the acceptance-criteria gate.

4. 4.

   Review the cases as a business tester, not a proofreader. Sample every critical-path case fully and a third of the rest. You are hunting three defects: expected results two people could score differently ("works correctly"), steps that assume system knowledge a business tester lacks, and data that would not survive first contact (formats, mandatory fields, referential setup). Fix in the pack, and note the pattern — the same prompt fix improves next sprint's generation.

5. 5.

   Have the business owner sign the matrix, not the pack. The owner cannot meaningfully review 200 test cases; they can absolutely review a one-page matrix saying which requirements are covered, partially covered, and untested, with priorities visible. Their sign-off on the coverage position — including any accepted gaps, in writing — is the honest form of UAT scope agreement.

6. 6.

   Use the matrix as the live UAT dashboard. As defects arrive, ask the AI to cluster them: "map each defect to the requirement and test case it implicates; flag defects matching no test (escaped coverage) and requirements accumulating multiple defects." Escaped-coverage defects feed new test cases; defect concentrations feed the go/no-go narrative with evidence attached.

Make it yours

• Jira/Xray-class AI for pipeline-native teams: generate cases directly against stories so the traces-to links live in the tool; still export the coverage matrix — tool dashboards show execution status, and this matrix shows design coverage, which is the earlier and more important signal.
• Azure DevOps test plans: same pattern — generated cases land as work items linked to requirements, and a matrix query becomes a standing widget.
• Claude Project per release: stories, criteria, and the pack in one project; mid-UAT questions like "which tests touch the refund calculation?" get instant cited answers.
• Spreadsheet mode for tool-light teams: the whole pack as a workbook — cases on one tab, matrix on another with conditional formatting on NONE/PARTIAL — perfectly sufficient for a single release's UAT.

Quality check

• Every criterion has a coverage verdictCOVERED, PARTIAL with the gap named, or NONE — and every NONE was resolved or accepted in writing.
• Orphan tests were adjudicateddeleted as scope creep or promoted into a raised requirement; none silently retained.
• Negative and boundary cases existfor every criterion implying limits, authorization, or validation, not just the happy path.
• Expected results are score-proofsampled cases pass the "could two testers disagree on pass/fail?" test.
• Test data is synthetic throughoutrealistic formats, zero production personal data, referential setup included in preconditions.
• The business owner signed the matrixcoverage position, priorities, and accepted gaps, dated before UAT execution began.