BXUP Track progress & test yourself — free
Engineering Leadership · Hands-on lab · ~5 min read
ENGINEERING LEADERSHIP · HANDS-ON LAB

Lab 10 · Rollout Playbook With Kill Criteria

Phase an AI-tooling rollout for a 60-person org — pilot to cohort to scale, four named champion roles, an enablement plan, and per-phase success AND kill criteria — then survive the pressure test: a VP email demanding you skip the pilot and mandate org-wide use this sprint.

Lucia Ferreira is Director of Engineering at Halcyon Freight Software, a fictional 60-engineer org that has just signed its AI-tooling contract. Two teams are excited, one is hostile, and most engineers are waiting to see whether this is a tool or a fad. Lucia has watched a neighboring company "roll out" AI by all-hands announcement — 90% seat activation, 20% real usage, quiet abandonment within two quarters. She knows the difference between a mandate and a rollout is infrastructure: champions with actual roles, enablement with actual time, phase gates with actual criteria — including the criteria that would stop the rollout, written down before anyone is emotionally invested in continuing it.

You will need

  • A chat AIfree ChatGPT, Claude, or Gemini, for drafting and red-teaming.
  • A doc for the rollout plan, comms pack, and pushback note.
  • Timeabout 75 minutes.
  • No real datathe org and the VP email below are synthetic.

The inputs

HALCYON ORG SNAPSHOT
60 engineers, 7 teams. Teams Atlas and Beacon: enthusiastic early
users (some shadow usage already). Team Cinder: openly skeptical
after a bad agent demo corrupted a test environment in March.
Four teams: neutral. Platform maturity: decent CI, spotty test
coverage (41-83% by team). Review standards for AI code: drafted,
not yet adopted (see Lab 05). Measurement baseline: captured last
quarter (Lab 08 discipline). Budget: approved for all 60 seats.

THE VP EMAIL (arrives day 3 of your planning)
From: VP Product & Engineering
Subj: Let's go faster on AI
"Lucia - saw the Shopify memo and the coverage around it. They
made AI a baseline expectation for everyone, overnight, and
they're printing productivity numbers. We already paid for 60
seats. I want org-wide mandatory adoption announced this sprint -
pilots are how big companies slow-walk things. Every week we
phase this is a week of paid-for licenses sitting idle. Can you
send the all-hands announcement draft by Friday? - R"

CHAMPION CANDIDATES (from your skip-levels)
- Meera (Atlas, senior): heaviest real user, writes prompt guides
- Josh (Beacon, mid): built the team's eval spreadsheet unprompted
- Sana (Cinder, senior): the org's sharpest skeptic; respected
- Dev (Platform, staff): owns CI; quietly automated two workflows

Steps

  1. Design the three phases with entry and exit conditions. Pilot (2 teams, 4–6 weeks), cohort (3 more teams, 6 weeks), scale (all teams). For each phase write: who enters and why, what support exists, the success criteria that open the next phase, and the kill criteria that pause or stop it. Choose pilot teams deliberately: one enthusiastic (Atlas) and one neutral-with-good-baselines — not two fan clubs; a pilot that cannot fail teaches nothing.

  2. Write real kill criteria — numbers, windows, actions. Per phase, three to four conditions such as: change-failure rate rises more than 2 points above baseline for 3 rolling weeks → pause expansion, investigate; review burden per PR up more than 50% without churn improvement → pause; fewer than 40% of pilot engineers using the tool weekly by week 4 despite enablement → the tool or the enablement is failing, diagnose before cohort; any policy breach of the Lab 04 class → immediate scope freeze. Each criterion names its metric source and who calls it. Kill criteria written after launch are alibis; date-stamp yours.

  3. Define the four champion roles as jobs, not honors. Name them from your candidate list and write two lines each: Practice lead (Meera — owns prompt patterns and the weekly office hour), Measurement lead (Josh — owns the paired-metrics panel and the phase-gate evidence), Skeptic-in-residence (Sana — paid to find failures early, owns the risk log, veto-armed on phase gates), Platform lead (Dev — owns tool config, CI integration, sandbox safety). Champions get protected hours (4/wk) or the roles are decorative. Note the deliberate move: recruiting the org's sharpest skeptic into a real oversight role converts distrust into the rollout's immune system.

  4. Build the enablement plan. Per phase: onboarding sessions (2 hrs), role-specific patterns (not generic vendor demos), the review standards from Lab 05 adopted before pilot merge volume rises, weekly office hours, a shared prompt-and-failure log (failures are enablement gold), and protected learning time (2 hrs/wk during pilot). This is the "enablement infrastructure" the pushback note will lean on — build it before defending it.

  5. Write the comms pack: kickoff message. To the whole org at pilot launch: why these two teams first, what success and failure look like (say the kill criteria out loud — publishing them is what makes the rollout credible to skeptics), the timeline to everyone else, and where shadow users stand (amnesty plus an invitation into the office hours, not a scolding).

  6. Write the comms pack: FAQ and skeptic 1:1 script. FAQ: is this mandatory (not yet; by scale phase, expected-with-exceptions per Lab 03 logic), what about the March incident (what changed since), will metrics be used against individuals (no — team-level only, in writing). The 1:1 script for Cinder-team conversations: open by validating the March evidence, show the sandbox rules that now exist, offer the skeptic role — do not ask for enthusiasm, ask for scrutiny.

  7. Now answer the VP. Write the pushback note — one page, sendable Friday in place of the announcement the VP asked for. Requirements: agree with the goal (full adoption, fast) and refuse the method (mandate-by-announcement) with evidence; correct the Shopify reading specifically — what that case actually demonstrates is that the memo was the visible part of a system of enablement infrastructure (tooling access, training, usage expectations built into reviews, executive modeling of use), and copying the memo without the infrastructure copies the press release, not the result; convert "idle licenses" into the real waste comparison (six weeks of phased licenses vs a failed org-wide launch that poisons adoption for a year); give the VP a better fast lever — pull cohort forward if pilot gates pass early, and put the VP's name on the kickoff message; commit to a date for the org-wide decision with evidence attached.

  8. Red-team the plan. Prompt: "Find the three most likely failure modes of this rollout plan." Patch what you accept. Common finds worth pre-empting: champions' protected time evaporating under delivery pressure (mitigation: named in their sprint capacity), pilot success criteria gamed by enthusiasm (mitigation: Josh's paired metrics + Sana's veto), the VP announcing anyway (mitigation: the Friday note offers the co-branded kickoff — give the VP a stage that isn't a mandate).

Remember:

A rollout plan without kill criteria is a commitment to call whatever happens a success — and everyone in the org can smell that from the kickoff email onward.

Verify your work

A complete lab has all of the following:

  • Three phases with entry conditions, exit criteria, and 3–4 numeric, date-stamped kill criteria each, with named metric sources and a named decider.
  • Four champion roles filled from the candidate list with real duties and protected hours — including the skeptic role with gate veto.
  • An enablement plan sequenced so review standards land before pilot volume.
  • A kickoff message that publishes the kill criteria and handles shadow users with amnesty.
  • A skeptic 1:1 script that validates the March evidence before asking for anything.
  • A pushback note that reads as an acceleration plan, corrects the Shopify inference, reframes the license-idle argument, and offers the VP a visible role.
▸Answer key — the pressure test and the plan's load-bearing choicesoptional deep dive — click to expand

The planted pressure test. The VP email bundles four seductive-but-wrong arguments, and the pushback note is graded on countering each specifically. (1) "Shopify did it overnight" — the case actually shows a memo sitting on top of enablement infrastructure: tool access for everyone, learning time, AI expectations wired into real processes, and leadership using the tools themselves. The memo is the artifact that went viral; the infrastructure is what worked. Announcing the memo without building the machine produces seat activation, not adoption — the neighboring-company failure in the scenario preamble is the control group. (2) "Pilots are slow-walking" — pilots with dated exit criteria and a pull-forward clause are the opposite: they are how you go fast twice — first with two teams, then with evidence. Offer the acceleration explicitly: gates passed early → cohort starts early. (3) "Idle licenses are waste" — six weeks of partially idle licenses costs a few thousand dollars; a failed mandatory launch costs the adoption curve itself, plus the credibility of the next initiative. Name both numbers. (4) "Send the announcement by Friday" — send the pushback note Friday with the kickoff message attached, VP co-signed, pilot launching Monday. Never answer a demand for speed with a request for patience; answer it with a faster-looking plan that is actually safer.

Load-bearing plan choices a strong answer makes. Pilot pairs an enthusiastic team with a neutral one (a two-fan-club pilot cannot generalize). Sana is recruited, not managed around — the skeptic role with real veto power is both the best failure-detector and the most persuasive signal to the hostile team. Review standards adopt before pilot volume rises (retrofitting standards after merge volume doubles is how the March incident recurs at production scale). Kill criteria are published in the kickoff — secrecy about failure conditions reads as bad faith and forfeits the credibility that makes scale-phase "expected use" acceptable later.

Kill-criteria quality bar. Each has metric + threshold + window + source + named decider + action. "We'll keep an eye on quality" fails. "CFR +2pts over baseline for 3 rolling weeks (Josh's panel) → expansion pause, investigation owned by Dev, restart requires gate re-pass" passes.

Grading yourself. Pushback note counters all four arguments and offers the pull-forward lever: strong pass. Note argues "trust me, pilots are best practice" without the Shopify correction or the waste reframe: it will lose the room — redo step 7. Plan has success criteria but no kill criteria, or kill criteria without deciders: the rollout cannot be stopped by its own design, which is the exact failure the lab exists to prevent.

Key Takeaway:

Rollouts succeed on infrastructure — champions with hours, enablement before volume, gates with numbers — and stay honest through kill criteria written before launch. When the mandate email arrives, the answer is not resistance; it is a faster plan with brakes.

This is the free reading view. Open BXUP to track your progress, run the hands-on labs, and prove it with original, exam-pattern assessments.
© Anvi Innovate · BXUP — Be Exponentially Upskilled · BXUP is an independent learning platform, not affiliated with or endorsed by AWS, Microsoft, Google, NVIDIA, IBM, Salesforce, Databricks, Oracle or Anthropic. All trademarks belong to their respective owners. · bxup.in