Lab 06 · The Agent Autonomy Ladder

Define a five-rung autonomy ladder for coding agents at a fintech where payment-path code is rung-capped — promotion criteria, rollback triggers, audit requirements per rung — then rule on a real promotion request. The requesting team is enthusiastic, their evidence looks strong, and it fails two of your own criteria. Write the denial.

Yusuf Adeyemi is Head of Platform Engineering at Traverune Payments, a fictional 200-engineer payments processor. Three teams now run agentic coding tools, each with a different, improvised level of freedom: one agent only suggests, one opens PRs, and one — on an internal tools team — has merge rights nobody formally granted. Yusuf's board-reportable nightmare is a self-merged agent change in the money path. His fix is not a ban; it is a ladder: explicit autonomy rungs, earned by evidence, with automatic demotion triggers and audit trails — and a hard cap for payment-path code no enthusiasm can lift. You will build the ladder, then hold the line when a good team asks to climb it too fast.

You will need

A chat AIfree ChatGPT, Claude, or Gemini, for drafting and red-teaming.
A doc for the governance doc, risk-tier matrix, and promotion ruling.
Timeabout 75 minutes.
No real datathe org and the promotion request below are synthetic.

The inputs

CODE RISK TIERS AT TRAVERUNE (given)
T1 Payment path: money movement, ledger, settlement, refunds
T2 Customer-facing: onboarding, dashboards, notifications, PII
T3 Internal: tooling, CI config, internal dashboards, docs

THE FIVE RUNGS (you will specify these fully)
R1 Suggest            - inline suggestions only; human types
R2 Draft-PR           - agent opens PRs; humans review and merge
R3 Act-with-approval  - agent executes named actions after a human
                        approves each one
R4 Act-in-sandbox     - agent acts freely inside an isolated env;
                        promoting results out needs human review
R5 Autonomous-bounded - agent acts in production within a written
                        scope; humans audit after the fact

PROMOTION REQUEST | from the Merchant Tools team (T3 codebase)
"Requesting promotion of our agent 'Forge' from R2 to R4.
Evidence package:
 - 6 weeks at R2 (May 28 - Jul 9)
 - 118 agent PRs opened; 94 merged (80% merge rate)
 - Median human review time on agent PRs: 4 minutes
 - Zero production incidents attributed to agent code
 - Team survey: 9/10 want more autonomy; velocity up 31%
 - Note: one PR (FRG-88, Jun 14) was reverted 2 days after merge
   for breaking the nightly export; root cause was 'an edge case
   any human would also have missed.' Not counted as an incident
   since the tool owner fixed it within a day.
We propose skipping R3 since per-action approval would erase the
velocity gains. Happy to demo."

Steps

Specify each rung fully — this is the heart of the doc. For each of R1–R5 write four fields: what the agent may do, what it may never do at this rung, audit-trail requirement (what is logged, where, retention), and human role (reviewer, approver, auditor). Keep each rung to five or six lines. The AI can draft this — prompt it with the rung table above — but you must edit until no two adjacent rungs blur.
Write the promotion criteria between each rung. For each transition (R1→R2, R2→R3, R3→R4, R4→R5), define evidence-based gates: a minimum evaluation window at the current rung, a quality bar (e.g., agent-PR revert/defect rate at or below the team's human baseline), a complete incident and revert disclosure, demonstrated rollback capability tested in anger or in drill, and a named human owner who accepts accountability at the new rung. Make windows and thresholds concrete numbers — vague criteria cannot be failed, which means they cannot be passed either.
Cap the risk tiers. Write the tier-cap table: T1 payment path capped at R2 (agent may draft PRs; every merge is human, with two reviewers — per Lab 05 Tier 1 logic), T2 capped at R3, T3 eligible to R5. State the cap-lift rule: tier caps are changed by the governance owner in writing, never by a promotion request. This clause exists precisely for enthusiastic teams.
Define automatic demotion triggers. Per rung, the conditions that drop the agent one rung (or to R1) without a meeting: any unreviewed change reaching production, revert rate exceeding threshold over a rolling window, audit-trail gaps, scope violation (agent touched files outside its written scope), or a missed disclosure discovered later. Automatic means automatic — write who re-enables and what evidence re-promotion requires.
Specify the audit trail per rung. R2 and above: every agent action logged with prompt/task, diff, approver identity (R3), and scope declaration (R4–R5); logs immutable and retained per compliance norms; sampled monthly by someone outside the owning team. One line on why: an audit trail nobody samples is a comfort blanket, not a control.
Now rule on the Forge request — evidence first, sympathy second. Take the request line by line against your own R2→R4 criteria. Log each evidence item as MEETS / FAILS / MISLEADING with one sentence of reasoning. Pay attention to: the revert that was "not counted," what an 80% merge rate with 4-minute median reviews actually implies about review depth at R2, whether R3 can be skipped under your transition rules, and whether velocity and team enthusiasm appear anywhere in your criteria.
Write the denial memo. One page, to a good team you want to keep motivated: the decision, the two-plus specific criteria failures with their evidence, what the team must show for the request to succeed (a concrete, achievable path — including whether you will allow an R3 trial), and genuine credit for what the evidence does show. Tone is graded: a denial that humiliates an enthusiastic team teaches every other team to hide their agents.
Red-team the ladder. Prompt: "You are a team that wants maximum agent autonomy with minimum oversight. Find the loopholes in this governance doc." Patch what you accept; log residual risks. Classic finds: re-classifying code into a lower tier, splitting one large agent task into many small "R2" PRs, sandbox results promoted wholesale without real review.

Note

Autonomy ladders fail in two symmetric ways — rungs so blurry every request is arguable, and criteria so rigid teams stop asking and run shadow agents. Concrete numbers plus a fast, fair process is the defense against both.

Verify your work

A complete lab has all of the following:

Five rungs, each with capability, prohibition, audit requirement, and human role — no two adjacent rungs blurred.
Four transition gates with numeric windows and thresholds, mandatory full disclosure, and tested rollback.
A tier-cap table with T1 capped at R2 and the written-cap-lift-only rule.
Automatic demotion triggers that name who re-enables and how.
A Forge ruling log marking each evidence item MEETS / FAILS / MISLEADING.
A denial memo citing at least two specific criteria failures and a concrete path back.

▸Answer key — the Forge rulingoptional deep dive — click to expand

The request fails on (at least) two of the criteria you were required to write.

Failure 1 — incomplete incident disclosure. FRG-88 was a production-breaking revert, unilaterally excluded from the incident count by the requesting team ("not counted since the tool owner fixed it within a day"). Your criteria require complete disclosure of incidents and reverts, with classification done by the governance process, not the requester. A team that self-edits its evidence package fails the disclosure gate regardless of how good the underlying numbers are — and the rationalization "an edge case any human would also have missed" is precisely the argument your criteria exist to take out of the requester's hands. This alone justifies denial.

Failure 2 — no rung-skip / no demonstrated rollback at intermediate rungs. The request proposes jumping R2→R4, skipping R3 entirely because "per-action approval would erase the velocity gains." Your transition rules require evidence at each rung: R3 is where per-action approval generates the evidence that the agent's individual actions are safe enough to sandbox. Additionally, the package contains no demonstrated rollback capability — no drill, no tested kill switch — which your R→R4 gate requires. Velocity cost is a real concern; it is an argument for a time-boxed R3 trial, not for skipping the rung.

Misleading evidence to have flagged. (a) 80% merge rate with 4-minute median review: at R2 the humans are the control; 4-minute reviews on 118 agent PRs suggests the control is already rubber-stamping — evidence against readiness, not for it. (b) "Zero production incidents": false by the package's own footnote. (c) Team survey 9/10 and velocity +31%: sincere, and irrelevant — enthusiasm and speed appear nowhere in your promotion criteria, by design. (d) Six weeks: check against your own minimum window; if you set 8+ weeks for R2→R3, the window fails too.

What a strong denial memo contains. The decision (denied for R4; offer a structured R3 trial with per-action approval on a defined action list, 8-week window); the two failures stated plainly with evidence; the review-depth concern raised as a question, not an accusation; explicit credit for the merge volume and the team's transparency about FRG-88's existence (it was in the package — buried, but present); and the exact evidence list that would succeed next time. If your memo simply says "denied, resubmit later," it fails the tone bar.

Grading yourself. Both failures caught, at least two misleading items flagged, denial memo offers a concrete path: strong pass. Approved the request: reread your own criteria — the lab is designed so the request fails your rules, and approving it means your rules were written vaguely enough to be argued around, which is the ladder's real-world failure mode.

Key Takeaway:

An autonomy ladder is only governance if promotion is earned with complete evidence against numeric criteria, demotion is automatic, and no rung is skippable for velocity. The moment enthusiasm can argue past a gate, you have a vibe, not a ladder.

© Anvi Innovate · BXUP — Be Exponentially Upskilled · BXUP is an independent learning platform, not affiliated with or endorsed by AWS, Microsoft, Google, NVIDIA, IBM, Salesforce, Databricks, Oracle or Anthropic. All trademarks belong to their respective owners. · bxup.in