Take a month of bot conversation logs, compute the vendor-style 68% deflection number, then apply the strict definition — fully resolved, no human touch, no reopen within 72 hours — watch it become 34%, and write the one-page memo that explains the gap to a CIO without killing the project.
Nisha's desk deployed a virtual agent six months ago, and the vendor dashboard says 68% deflection. Her CIO wants to cut two L1 positions on the strength of it. Nisha has read the conversation logs, and she knows what the dashboard counts: users who gave up, users who got a KB link and raised a ticket anyway, users whose "resolved" issue reopened two days later. This lab is the measurement discipline that protects both the desk and the truth: compute the vendor number honestly, compute the strict number honestly, reconcile them line by line, and present the gap as maturity rather than failure — because a desk that reports 34% with a straight face is more credible than one that repeats 68% and gets found out at headcount time.
The data below is seeded with every standard inflation mechanism. Your job is to find them all and price each one.
~60 minutes hands-on · Core tier (maps to module M7). $0 — a spreadsheet or plain markdown table, plus any free AI chat for drafting the memo (with every number pinned by you, not it).
Save the month's aggregate export as bot-month.txt:
MONTH: July 2026 | Channel: portal + Teams
Total bot sessions ................................ 1,240
Sessions vendor marks "contained" (no live handoff) ..: 843
of which: user abandoned mid-conversation ........: 217
of which: bot answered, user got KB link only ....: 391
of which: bot answered, marked resolved by user ..: 235
Sessions escalated to human in-session ...........: 397
Tickets raised by same user within 24h of a
"contained" session (portal/email/walk-up) ......: 154
of which traced to: KB-link-only sessions .......: 118
of which traced to: user-marked-resolved ........: 21
of which traced to: abandoned sessions ..........: 15
User-marked-resolved sessions reopened within 72h .: 38
BSAT survey (sent after "contained" sessions,
response rate 11%): avg 3.1/5; among users who
later raised a ticket: 1.9/5
Password-reset auto-resolutions included above ....: 96
(of which 9 were the same 3 users, repeatedly)
And a 12-row sample of individual "contained" sessions as session-sample.txt — the ground truth beneath the aggregates:
S-0107 KB link sent (VPN). User raised ticket 3h later.
S-0142 Password reset flow completed. No recontact.
S-0231 User: "never mind ill just email you". Abandoned.
S-0288 Bot answer accepted, marked resolved. No recontact.
S-0301 KB link sent (MFA). No recontact observed.
S-0356 Marked resolved; same issue reopened day 2.
S-0399 User asked 3x, rephrased, closed tab. Abandoned.
S-0410 Password reset for same user, 3rd time this month.
S-0455 Bot answer accepted, marked resolved. No recontact.
S-0470 KB link sent (drive access). Walk-up to desk same day.
S-0521 Marked resolved. BSAT response: 1/5 "bot was useless".
S-0577 Bot answered, user replied "thats wrong", escalated.
Definitions to apply. Vendor deflection: contained sessions ÷ total sessions. Strict deflection: sessions where the issue was fully resolved, no human involvement at any point, and no reopen or same-user recontact on the same issue within 72 hours ÷ total sessions. Abandonment: abandoned ÷ total. Treat a ticket-within-24h or a 72h reopen as disqualifying; treat KB-link-only sessions with no recontact as resolved only if you can defend that assumption — and price the alternative.
measurement-workbook.md precisely what this definition counts: any session that did not hand off live, including people who gave up.cio-memo.md: the two numbers and the staircase between them; what the bot genuinely does well (password resets, MFA guidance — say so, with the chronic-repeat caveat); the headcount implication stated honestly (34% strict deflection concentrated in low-complexity contacts does not fund two L1 cuts); and the go-forward: fix the boomerang drivers (KB-link answers that do not resolve), instrument reopen tracking permanently, and re-baseline quarterly.The temptation in this lab is to weaponize the strict number and declare the bot a failure. That is the same dishonesty pointed the other way. The bot resolves hundreds of real contacts — the memo's job is to say what is true in both directions: less than the dashboard claims, more than a cynic would concede, and measurable from now on.
Self-grading rubric (pass = 7/10): vendor number reproduced then dissected (0–2) · all three inflation mechanisms found and priced (0–3) · KB-link treatment defended with a range (0–2) · memo honest in both directions with headcount implication (0–2) · AI-draft audit performed (0–1).
Deflection inflates three ways — surrender counted as success, delayed tickets counted as deflected, and weak resolved signals never re-checked — and the operator who can walk a CIO from 68% to 34% line by line, without cynicism and without spin, becomes the desk's most trusted voice on every AI number that follows.