Run the Change-Impact Assessment That Decides Whether Your AI Rollout Lands

Workflow area: AI Rollout & Governance · Tier: Advanced · Hands-on time: ~45 minutes

The MIT NANDA finding that ~95% of GenAI pilots show no P&L impact is mostly a change-management verdict: tools shipped into workflows nobody redesigned, to people nobody prepared, measured by license counts nobody uses. AI change differs from classic system change in three ways that standard change plans miss — the tool is probabilistic, so users must learn when to distrust it; the tool reshapes judgment work, which people experience as identity threat, not process change; and every AI-touched role acquires a new duty that appears in no old job description: verifying machine output. The BA who maps those three honestly writes the difference between adoption and shelfware.

When to use — and when not to

• Use it for any AI deployment that changes how people workwhich is every deployment beyond a personal productivity tool; run it during design (pb-46), not the week before go-live.
• Use it when the agent redesign assigns humans new rolesexception handlers, output reviewers, and escalation owners are new jobs wearing old titles; their task deltas are the core of the assessment.
• Use it when an earlier rollout stalledthe assessment run retrospectively diagnoses whether the blocker is trust, skill, workflow friction, or quiet rational resistance to a tool that makes the job worse.
• Skip the apparatus for opt-in personal toolinga voluntary drafting assistant needs training and guardrails, not an impact matrix; save the rigor for changes people cannot opt out of.
• Skip it as a communications plancomms is one output; if the assessment produces only newsletter copy and no workflow, job-aid, or metric changes, it produced nothing.
• Do not run it purely from the org chartimpact lives in tasks, not titles; without task-level input from the affected teams, the matrix is management fiction.

The recipe

1. 1.

   Build the task-delta matrix from the to-be process. For every affected role, walk the redesigned process (pb-46) task by task: what disappears, what changes, what is new — with the new almost always including verification and exception duties. Draft with AI, correct with the teams:

   Copy

   From the as-is and to-be process maps below, build a change-
   impact matrix: Role | Task | Change (REMOVED / MODIFIED / NEW)
   | What the person now does differently | New judgment required
   (when to trust vs verify vs escalate the AI) | Time impact
   estimate | Risk if performed badly
   Rules: verification and exception-handling duties are NEW tasks,
   never "the same task, faster". Flag any role whose removed tasks
   exceed its new tasks — that is a role-scope conversation for HR,
   not a training slide. Mark uncertain estimates [VALIDATE].
   Maps: [PASTE AS-IS + TO-BE]
   

   Then validate with the people who do the work — their corrections are the assessment, exactly as walkthrough corrections are the process map (pb-23).

2. 2.

   Diagnose the trust barriers per group, in both directions. Interview a sample of each affected group and classify what you hear: under-trust ("it will be wrong and I'll be blamed") and over-trust ("the AI checked it, so I don't have to") are both adoption failures — the second is the dangerous one, and it arrives later, after the tool has been right for three quiet weeks. For each barrier, note what evidence would shift it: seeing the eval scores, watching the escalation path work, a named human owner for mistakes. Those become plan items, not posters.

3. 3.

   Convert the matrix into skill deltas and verification-first job aids. The distinctive AI training need is calibration — when to accept, when to check, when to escalate — which classroom demos do not build. Design practice on real cases: each group works 10–15 golden-set cases (pb-48) including ones the AI gets wrong, so they experience the failure modes before production shows them one. Every job aid gets a verification section: what to check on every output, what only on flagged ones, and the escalation route with its SLA (pb-46's queues). A job aid without verification duties is an instruction to rubber-stamp.

4. 4.

   Define adoption metrics that cannot be gamed by logging in. License activations and prompt counts measure theater. Instrument the behavior chain instead: workflow adoption (share of cases actually processed through the new path), calibrated usage (override rate — near-zero means rubber-stamping, very high means distrust; both trigger investigation, with healthy bands set from the pilot), outcome linkage (the charter's metrics, pb-42, cut by team), and experience (short pulse on "the tool makes my job better/worse" — the honest leading indicator of quiet abandonment). Publish the bands before go-live so nobody retro-fits targets to actuals.

5. 5.

   Assemble the plan with a feedback loop that visibly acts. Sequence: pre-launch (role conversations for scope changes, calibration training, job aids in hand), launch (floor-walking support, daily exception-queue triage in week one), and the standing loop — a channel where users report bad outputs, weekly triage feeding the golden set and prompt fixes, and visible "you reported X, we changed Y" notes. The loop is the highest-leverage line in the plan: trust follows demonstrated responsiveness, and the reports are your production eval data arriving free.

Make it yours

• Survey-plus-interview at scale: run the trust diagnosis as a short instrument across the whole population, then interview the outliers; the distribution per team tells you where to spend launch support.
• Change-impact one-pager per role: distill each role's row of the matrix into a single page — what changes for you, what's new, where to get help — reviewed by that team's lead before distribution.
• Works-council and HR early: where task deltas change role scope or measured performance, involve employee representatives at the matrix stage, not the announcement stage.
• At program scale: standardize the matrix and metric bands across rollouts, so the org learns which barrier patterns predict stalls and which interventions actually move them.

Quality check

• Task deltas are validated by the people affectedevery row either carries a team's confirmation or a [VALIDATE] flag; org-chart-only analysis is not baselined.
• Verification duties are explicit, timed, and staffednew checking work has time estimates in the matrix and headcount consequences in the business case (pb-38), not hidden inside "efficiency gains".
• Both trust failures are addressedthe plan contains specific counters to over-trust (seeded-error awareness, sampled QA) as well as under-trust.
• Training is calibration practice on real casesincluding failure cases; a slide deck alone fails this check.
• Adoption metrics have healthy bands, set in advanceoverride rates and workflow-adoption shares with investigation triggers, published before launch.
• The feedback loop demonstrably closesreported issues traceably become fixes or golden-set cases, and users can see that happen.