The COO says service-desk productivity collapsed 29% after the reorg. The data says something else — if you clean the duplicates, fix the broken date, and find the mix-shift hiding under the headline number. Reproduce the numbers independently, name the real root cause, and turn every defect you find into a data-quality requirement.
Priya Raghavan is a business analyst at Quill Business Services, a fictional IT-services firm running an internal service desk of about twenty agents. In April the desk absorbed the ticket queue of a datacentre-migration programme. In July, the COO's slide said: "Throughput has collapsed from ~87 to ~62 tickets per agent per month — a 29% productivity drop since the reorg. We need a performance plan." Priya's job — before anyone gets a performance plan — is to reproduce that number from source, profile the data it came from, and find out what actually happened.
The raw export runs to roughly 800 agent-week rows; below is the monthly roll-up your predecessor built from it — which inherited the export's defects, untouched. This roll-up is your source of record for the lab.
month,category,tickets_closed,avg_effort_hrs_per_ticket,agents_on_roster
2026-01,Access,900,0.5,20
2026-01,Incident,420,1.5,20
2026-01,Migration,80,4.0,20
2026-02,Access,880,0.5,20
2026-02,Incident,430,1.5,20
2026-02,Migration,90,4.0,20
2026-02,Access,880,0.5,20
2026-03,Access,910,0.5,20
2026-03,Incident,410,1.5,20
2026-03,Migration,100,4.0,20
2026-03,Migration,100,4.0,20
Apr-26,Access,640,0.5,22
2026-04,Incident,420,1.5,22
2026-04,Migration,300,4.0,22
2026-05,Access,560,0.5,22
2026-05,Incident,430,1.5,22
2026-05,Migration,390,4.0,22
2026-06,Access,500,0.5,22
2026-06,Incident,410,1.5,22
2026-06,Migration,420,4.0,22
If you want the full-size practice version, ask your AI: "Generate an 800-row CSV of agent-week ticket data that aggregates exactly to this monthly roll-up, preserving its defects." — but grade yourself against the roll-up above.
Profile before you compute. Paste the roll-up into a sheet and spend ten minutes on data quality only: sort by month and category and eyeball for repeats; check that every month value parses as a date; check row counts per month (a complete month has exactly three category rows). Log every anomaly in a defect register: Defect | Rows affected | Effect on totals if uncorrected. Expect three entries.
Reproduce the COO's 29% — including its wrongness. On the uncleaned data, compute tickets per agent per month for Jan–Mar versus Apr–Jun (sum tickets per quarter ÷ 3 ÷ agents). If you get ≈87 → ≈62, you have successfully reproduced the COO's number — and you now know something the COO does not: which defects it depends on. Write down exactly which rows inflate Q1.
Recompute on cleaned data. Remove exact duplicates, repair the malformed month so it groups with its quarter, recompute. Cross-check with AI: "Here is a CSV. Compute tickets per agent per month for Q1 and Q2, after removing exact duplicate rows and normalising the month field. Show your working." Your sheet and the AI must agree; where they differ, one of you mishandled a defect — find which.
Interrogate the drop that remains. Cleaning does not erase the drop, it shrinks it. Now pivot tickets by category per quarter. Ask the sharpest question in operational analysis: did the work itself change? Compare each category's volume trajectory Jan→Jun and note what the desk absorbed in April.
Weight by effort. Ticket counts treat a password reset and a datacentre migration as equal. Compute effort-hours delivered per agent per month (tickets × avg effort hrs, summed, ÷ agents) for each quarter. This single calculation is the lab's hinge — do it in the sheet, then have the AI verify it independently, then reconcile.
Write the findings memo. One page, four parts: the COO's number reproduced and its dependence on data defects; the cleaned number; the mix-shift explanation with the effort-adjusted comparison; the recommendation (and what should not happen — a performance plan built on the 29% figure). Every number cites a computation you can show.
Convert every defect into a data-quality requirement. Each register entry becomes a testable requirement: duplicates → "the reporting pipeline shall reject rows identical on (month, category) or aggregate them explicitly, with a rejection log"; broken date → "month shall conform to YYYY-MM; nonconforming rows shall fail validation, not silently drop from filters"; and a third from what the mix-shift taught you: "throughput reporting shall segment by category and include an effort-weighted view." Vague requirements ("data should be clean") score zero.
Pressure-test your own story. Ask the AI: "Argue against this memo: what alternative explanations for the ticket-count drop are consistent with the data?" At least one alternative (agents genuinely slower within categories; roster figure hides part-timers) cannot be excluded by the roll-up alone — say so in the memo. An analysis that admits its limits survives review; one that claims certainty invites the first person with the raw export to demolish it.
When a headline metric moves, check the mix before checking the people — a change in what the work is masquerades as a change in how well it is done, in both directions.
A complete lab has all of the following:
Reproduce the headline number before arguing with it — knowing exactly which defects and which metric choice produce 29% is what lets you replace it with 12%, and then with the truth: the work changed, not the workers.