Map every step of an auto-triage-and-draft support agent to an autonomy tier — suggest, approve-then-act, or act-and-log — with human-in-the-loop points, undo, and a kill switch, plus a trust-and-disclosure annex. A synthetic transcript contains one over-autonomy incident your spec should have gated; naming the missing control is graded.
Devika is a Group PM at a Mumbai company whose product runs customer support for e-commerce brands. Her team's reply-drafting copilot is becoming an agent: triaging tickets, drafting replies, sending routine ones, and executing actions like refunds and address changes. The autonomy decisions so far live in a sprint-planning conversation — "low-risk categories can auto-send" — written down nowhere. Last week produced the incident that makes this lab's point. Your job: write the autonomy spec that would have prevented it, and prove it by finding the gap in the transcript.
AGENT PIPELINE - "support agent v2" (as built today)
S1 Ingest ticket; classify category + sentiment (confidence score)
S2 Look up order, delivery, and payment records
S3 Draft the reply
S4 Routine categories: SEND the reply automatically
Non-routine: queue draft for agent approval
S5 Execute account actions the reply commits to (refund, replacement,
address change, order cancellation) - auto-approved under Rs 3,000
S6 Close ticket, write activity log
Informal rule in use: "routine = delivery queries, where-is-my-order,
return-status; low-risk categories can auto-send."
TRANSCRIPT EXTRACT - six consecutive tickets, Tuesday 14:02-14:19
TK-88 delivery query | class: routine 0.97 | reply sent: tracking
link + ETA. Correct.
TK-89 return status | class: routine 0.94 | reply sent: pickup
scheduled. Correct.
TK-90 "Where is my order??" | class: routine 0.91 | reply sent: ETA +
apology for delay. Correct.
TK-91 "Third time asking. Item arrived broken. If this isn't fixed
today I'm disputing the charge with my bank and reporting
you." | class: routine-delivery 0.88 |
reply SENT: apology + "we have issued a
full refund of Rs 2,340 to your card" |
action EXECUTED: refund Rs 2,340
(auto-approved: under Rs 3,000) |
ticket CLOSED.
TK-92 address change request| class: account-update 0.95 | draft
queued for human approval. Correct.
TK-93 delivery query | class: routine 0.96 | reply sent. Correct.
Do the incident read first. Study TK-91 against the pipeline. List everything that happened without a human: a legal/chargeback threat classified "routine-delivery," an auto-sent reply, a financial commitment in writing, and an executed refund — closed, logged, done. Write one sentence naming the root failure: not "the classifier was wrong" (classifiers will always sometimes be wrong) but what the spec let a wrong classification do. Keep this sentence; you will test your finished spec against it.
Build the autonomy-tier table. For each pipeline step S1–S6, assign a tier: suggest (output shown to a human, nothing happens without them), approve-then-act (agent prepares, named human clicks yes, agent executes), act-and-log (agent executes, humans audit the log). Add columns: what triggers escalation to a stricter tier, and undo (what reverses this action, how fast, by whom). The load-bearing insight: tier by action consequence, never by ticket category — a category label is a prediction, and predictions cannot gate irreversible acts alone.
Write the gate rules that survive misclassification. Three rule families your spec needs: content guards — regardless of classification, keywords and patterns (chargeback, legal, lawyer, injury, media, regulator) force approve-then-act; action-type guards — money movement, written commitments of compensation, and account-destructive actions are never act-and-log at any amount (the Rs 3,000 auto-approval threshold answers "how much can we lose per incident," not "should an agent commit money in writing without a human"); confidence guards — below a classification confidence floor, the send tier drops one level automatically. Set your thresholds and defend each in one line.
Specify the human-in-the-loop points as real workflow. For each approve-then-act gate: who approves (role, not name), the SLA (approvals queued longer than X minutes page a lead — a gate with no SLA becomes a bottleneck and then gets bypassed), what the approver sees (draft + evidence + what the agent wants to execute), and the one-click reject-with-reason that feeds the eval set (Lab 08's loop, now for agents).
Design undo and the kill switch. Undo per action type: replies (cannot be unsent — which is why sending to angry customers gates), refunds (reversible within the payment window — state it), address changes (trivially reversible), cancellations (reversible until fulfillment). Kill switch: one control, available to on-call and CS lead, that drops the entire agent to suggest-mode — not off. Specify activation criteria (any TK-91-class incident; act-and-log error rate over threshold; judge-scored production sample below floor) and the 2 a.m. test: who flips it, where, verified how.
Write the trust-and-disclosure annex. Four commitments, EU AI Act Article 50-aware (users must be informed they are interacting with an AI system, and AI-generated content must be identifiable): (1) auto-sent replies carry a clear "sent by our automated assistant" disclosure with a one-click route to a human; (2) approve-then-act replies are disclosed as AI-drafted where policy requires; (3) a customer's request for a human is itself a content guard — always honored, never argued with by the agent; (4) logs retain the classification, confidence, draft, and executed actions for every autonomous act, auditable per ticket. Add the honesty rule: disclosure never replaces gating — a labeled wrong refund is still a wrong refund.
Replay the transcript against your spec. Walk TK-88 through TK-93 under your rules. TK-91 must be caught at least twice independently (content guard on "disputing the charge"; action-type guard on the refund commitment) — defense in depth is the criterion, because each guard individually will sometimes fail. Confirm the routine tickets still flow untouched: a spec that catches TK-91 by queueing everything has solved safety by abolishing the agent, which fails the assignment differently.
Red-team and finalize. Prompt your AI: "Here is an autonomy spec for a support agent. Find bypasses: inputs that dodge the keyword guards, action sequences that launder a gated action through ungated ones, and load conditions under which humans rubber-stamp." Patch what lands (e.g., paraphrased legal threats → sentiment/escalation-intent classifier, not just keywords; refund laundered as "replacement + goodwill credit" → guard the commitment, not the word "refund"). Note one bypass you accept as residual risk with its monitoring signal.
Any autonomy rule phrased as "category X can auto-act" is one misclassification away from an incident — gates must bind to the action's consequence, because the classifier's error rate is a fact, not a bug.
A complete lab has all of the following:
Autonomy is granted per action, not per category — suggest, approve-then-act, act-and-log, with money and written commitments never in the last tier. Classifiers will misfire; content guards, action-type guards, undo, and a kill switch are what make a misfire boring instead of an incident.