Synthesize UAT Results into a Go/No-Go Evidence Pack

Workflow area: UAT & Testing · Tier: Foundation · Hands-on time: ~30 minutes

The last week of UAT is where analysis quality gets tested in public. Someone has to stand in front of the sponsor and say "we are ready" or "we are not," and that statement is only as good as the synthesis behind it. Done by hand, the synthesis is a late night of pivot tables; done carelessly with AI, it is a confident paragraph that glosses over the two severity-2 defects everyone will remember later. The fix is a structured evidence pack: AI aggregates and drafts, every claim links to a test result or defect ID, and the recommendation section is written — and owned — by a human.

When to use — and when not to

• Use it at the end of every UAT cyclemid-cycle checkpoints and the final readiness review both need the same shape: what was planned, what ran, what passed, what is still open, and what that means.
• Use it when the go/no-go audience is mixedsponsors want a risk narrative, delivery leads want defect detail, and one evidence pack with layered sections serves both without two separate late nights.
• Use it when known issues will shipa go decision with open defects is defensible only if each one carries a documented severity, workaround, and business-impact statement someone signed.
• Skip it if the test results are not yet trustworthysynthesis cannot fix execution gaps; if half the results are "not run," the honest deliverable is a completion plan, not a readiness narrative.
• Skip the AI drafting for the recommendation itselfthe model can summarize evidence, but "we recommend go" is a human judgment carrying human accountability. Write that paragraph yourself.
• Skip it for trivial releasesa config change with six test cases needs a table in an email, not an evidence pack.

The recipe

1. 1.

   Export the raw evidence. Pull the test-execution report (case ID, requirement link, status, tester, date) and the defect list (ID, severity, status, affected requirement, workaround) from Jira, Xray, Azure DevOps, or whatever your program runs. CSV is fine. Add the UAT entry criteria and the agreed exit criteria — the pack must be judged against what was promised, not against vibes.

2. 2.

   Run the structured synthesis. Paste the exports into Claude or ChatGPT with this prompt:

   Copy

   You are a business analyst preparing a UAT readiness evidence pack.
   Release: [NAME AND SCOPE, ONE LINE]
   Exit criteria agreed at UAT kickoff: [PASTE THEM]
   
   From the test execution export and defect export below, produce:
   
   1. COVERAGE — planned vs executed vs passed, overall and per
      requirement area. Flag any requirement with zero executed tests.
   2. DEFECT POSITION — open defects grouped by severity. For each
      sev-1/sev-2: ID, one-line description, affected requirement,
      documented workaround (or NONE), age in days.
   3. EXIT-CRITERIA SCORECARD — each criterion: MET / NOT MET / AT
      RISK, with the specific numbers that justify the rating.
   4. RISK OF KNOWN ISSUES — for each defect proposed to ship: what
      breaks in business terms, who is affected, workaround cost.
   5. OPEN QUESTIONS — anything the data cannot answer.
   
   Rules: every number must be computable from the exports — show the
   arithmetic for coverage percentages. Cite defect and test-case IDs
   for every claim. Do NOT write a recommendation section. If data is
   missing or ambiguous, say so under OPEN QUESTIONS instead of
   estimating.
   
   Test execution export:
   [PASTE]
   Defect export:
   [PASTE]
   

3. 3.

   Audit the numbers before you trust the prose. Good output looks like this:

   Copy

   EXIT-CRITERIA SCORECARD
   - "95% of planned cases executed": MET — 412 of 430 executed
     (95.8%). 18 not-run cases all in Reporting (REQ-44..47).
   - "Zero open sev-1": MET — DEF-201 (sev-1) closed 03 Jun, retest
     passed (TC-388).
   - "No sev-2 without workaround": AT RISK — DEF-214 (sev-2,
     invoice rounding on multi-currency, REQ-31) has workaround
     NONE. Age: 9 days.
   
   RISK OF KNOWN ISSUES
   - DEF-214: finance users manually correct ~40 invoices/month
     until fixed; month-end close at risk if unfixed by 30 Jun.
   

   Recompute at least the headline coverage figure and every sev-1/sev-2 count directly from the export. A synthesis with one wrong number loses the room for all the right ones.

4. 4.

   Write the recommendation yourself. Below the scorecard, add the human section: your readiness assessment, the conditions attached to a go (fix-by dates, monitoring, workaround owners), and what a no-go would cost. Name it clearly: "Analyst assessment — decision owner: [business owner's name]."

5. 5.

   Walk the business owner through it before the meeting. The go/no-go forum is the wrong place for the decision maker to see DEF-214 for the first time. A 20-minute pre-read with the owner turns the meeting into a confirmation, and their corrections into the final pack.

Make it yours

• Claude Project per release: keep exit criteria, prior cycle packs, and the prompt in a Project; each cycle's synthesis then trends against the last one ("open sev-2 count: 5 → 2 → 1") without re-explaining context.
• Jira/Xray or ADO native reports first: let the tool compute execution and traceability numbers it already knows, and use the LLM for the parts tools are bad at — the risk narrative and the exit-criteria argument.
• Excel + LLM for tool-poor programs: when UAT lives in spreadsheets, have the model draft the pivot logic, then paste computed totals — not raw rows — into the synthesis prompt to keep numbers auditable.
• Template it for the program: freeze the five-section structure as your program's standard readiness pack so cycle 3 is comparable with cycle 1 and the steering committee learns where to look.

Quality check

• Every number recomputescoverage percentages and defect counts match the raw exports when you check them by hand; arithmetic is shown, not asserted.
• Every claim carries an IDtest case, defect, or requirement. A readiness statement without an ID is an anecdote.
• Exit criteria are quoted verbatimthe pack judges the release against the criteria agreed at kickoff, not a softened paraphrase.
• Known-issue risks are in business terms"rounding error on REQ-31" became "finance corrects 40 invoices a month"; a sponsor can weigh the second, not the first.
• The recommendation is signed and separatedclearly labeled as the analyst's assessment, with the decision owner named; the AI wrote none of it.
• Zero-coverage requirements are visibleanything with no executed test appears by name, because silent gaps are how "tested" releases fail in week one.