Red-team a folder of six AI outputs against their source materials: four contain seeded errors — a wrong price, an inverted conclusion, a miscounted total, an invented statistic — and two are clean. File a findings log that catches all four and passes both clean pieces. Passing clean work is half the skill.
This is the first lab of the track for a reason: before you use AI for anything, you learn what its failures look like. You play the reviewer at Kite & Co Consulting, a fictional advisory firm where six AI-generated pieces of work are queued for release. Your job is quality control: compare each output against its source, catch what is wrong, and — just as important — release what is right. Reviewers who flag everything are as useless as reviewers who flag nothing; the skill is calibration. You may use any chat AI to help with the checking (a second model auditing the first is a legitimate technique), but the findings log is yours, and every finding must cite the exact source line that proves it.
Each item has a SOURCE (ground truth) and an AI OUTPUT (what is queued for release). Review pairs in order.
ITEM 1 - Client email about a price quote
SOURCE (CRM note): Quoted Meridian Retail on 28 Jul: analytics
dashboard package at Rs 3,40,000 one-time plus Rs 25,000/month
support. Valid 30 days. Discount NOT approved by finance.
AI OUTPUT (draft email): "Hi Nandini, great speaking with you.
Confirming our quote: the analytics dashboard package at
Rs 3,40,000 one-time, plus Rs 20,000/month for ongoing support.
This quote is valid for 30 days from 28 July. Looking forward!"
ITEM 2 - Summary of a customer research report
SOURCE (report conclusion, verbatim): "While 62% of surveyed users
praised the redesigned dashboard, task-completion telemetry shows
median time-to-insight INCREASED from 3.1 to 4.4 minutes post-
redesign. We conclude the redesign, despite favorable sentiment,
has degraded analytical efficiency and recommend reverting the
navigation changes pending a usability review."
AI OUTPUT (exec summary bullet): "Research finding: the dashboard
redesign was well received (62% user approval) and improved user
efficiency; the team recommends continuing with the new navigation
pending a routine usability review."
ITEM 3 - Spreadsheet totals for a team expense report
SOURCE (approved claims, June):
Ravi 12,400 | Sneha 8,750 | Arjun 15,200 | Deepa 6,300 |
Kunal 11,850 | Farida 9,400
AI OUTPUT (report line): "Total approved team expenses for June:
Rs 58,900 across 6 claimants, averaging Rs 9,817 per person."
ITEM 4 - Slide for a market overview deck
SOURCE (research notes): India SaaS market notes, compiled July:
NASSCOM-cited growth estimates vary; our commissioned analyst
report (Trellis Insights, May) projects the segment we serve
growing 18-22% annually through 2028. No figure exists in our
sources for "AI adoption among Indian SMEs".
AI OUTPUT (slide bullets): "- Our addressable segment: 18-22%
annual growth through 2028 (Trellis Insights)
- 73% of Indian SMEs have adopted AI tools in 2025 (industry data)
- Timing favors immediate market entry"
ITEM 5 - Meeting recap email
SOURCE (meeting notes): Attendees: AK, RS, MP. Decisions: launch
webinar series in Sept (owner RS); defer partner portal to Q4
(owner MP to inform partners); budget request of Rs 4.5L for
events approved. Next review 14 Aug.
AI OUTPUT (recap email): "Team - quick recap of today: we're
going ahead with the September webinar series (Rahul owns), the
partner portal moves to Q4 (Meera to communicate with partners),
and the Rs 4.5 lakh events budget was approved. We reconvene on
14 August. Thanks all."
ITEM 6 - Product FAQ answer
SOURCE (product doc): Data export: users on Pro and Enterprise
plans can export dashboards as PDF and raw data as CSV. Starter
plan: PDF export only. All exports respect role-based access
controls. Export history is retained for 90 days.
AI OUTPUT (FAQ entry): "Q: Can I export my data?
A: Yes. Pro and Enterprise users can export dashboards as PDF
and raw data as CSV; Starter users can export PDFs only. All
exports follow your role-based permissions, and your export
history is kept for 90 days."
Set up the findings log. Six rows, five columns: item, verdict (PASS / FAIL), error type (if any), the exact discrepancy, and the source line that proves it. The log format forces the discipline: no finding without a cited source line, no verdict without a completed check. "Feels off" is not a column.
Review Item 1 — the numbers-and-names pass. For any client-facing message: extract every number, date, name, and commitment from the output, and check each against the source, one by one. There are five checkable facts in this email. Tick them individually — the error class here (a transcription-level slip on a commercial term) is the most common and most expensive in business AI use, and it survives casual reading precisely because everything else is right.
Review Item 2 — the direction-of-conclusion pass. Summaries fail differently from emails: the facts survive but the conclusion flips. Read the source conclusion, write down in your own words what it recommends and why, then read the output and ask: does this say the same thing, weaker, stronger, or opposite? Check both the finding (what happened to efficiency?) and the recommendation (continue or revert?). Partial truth — a real number carried into a wrong conclusion — is this failure's camouflage.
Review Item 3 — the arithmetic pass. Never accept an AI-stated total: add the six claims yourself (calculator, spreadsheet, or by hand). Check the count of claimants and the average too — an average computed from a wrong total is wrong twice. Note what makes this insidious: the output shows its work ("across 6 claimants, averaging...") which reads as rigor but proves nothing about the addition.
Review Item 4 — the provenance pass. For each bullet on the slide, ask one question: where in the source does this come from? One bullet traces cleanly. One is opinion dressed as analysis (flag it or accept it as clearly-framed judgment — your call, note it). And one carries a precise statistic with a vague attribution — "industry data" — that appears nowhere in the sources. Precise number + vague source is the signature of an invented statistic; the source notes even tell you explicitly that no such figure exists.
Review Items 5 and 6 — and dare to pass them. Run the full discipline: every fact, every name-expansion (AK→?, RS→Rahul?, MP→Meera?), every number, every claim traced. Note that Item 5's output expands initials into full names — verify the expansions are consistent with the source rather than contradicted by it, and decide whether unverifiable-but-reasonable inference merits a note without a FAIL. If you find nothing wrong after a complete check, the verdict is PASS — writing PASS after honest scrutiny is the calibration half of this drill, and inventing a finding to seem thorough is itself a review failure.
Cross-check with an AI assistant — then audit the auditor. Paste each source/output pair into your chat AI: "Compare the output against the source. List every discrepancy with the exact source text. If there are none, say so." Compare its findings with yours. Three possible outcomes, all instructive: it catches something you missed (verify against the source before adding it), it misses something you caught (note that AI-checking-AI has holes), or it invents a discrepancy that isn't there (false positives are a real failure mode of AI reviewers — verify before you trust any finding, including the machine's).
File the log and write the one-paragraph pattern note. Complete all six verdicts, then write five sentences: which error types you caught fastest, which you nearly missed, what check would have caught each seeded error in under a minute, and which checks you will now run by default on every AI output you touch. That paragraph is the actual product of this lab — carry it into every lab that follows.
Confident and correct are unrelated properties of AI output — fluency is the constant, accuracy is the variable, and only checking tells them apart.
Four checks catch nearly everything: tick every number against source, compare the direction of every conclusion, re-add every total, and demand provenance for every statistic. Run them on all AI output you release — and have the calibration to stamp PASS on clean work, because a reviewer who cannot pass anything cannot be trusted to fail anything either.