Team AI-Adoption Playbook: Policy, Review Gates, and Metrics That Survive Delivery

Tier: Advanced · Workflow area: Team Leadership

This playbook is for the tech lead or staff engineer writing the team's AI operating policy: sanctioned tools, provenance rules, review gates, and the metrics that tell you whether AI speed is surviving contact with delivery.

When to use

• •
  You lead a team where AI usage is already universal but ungoverned — every engineer has a different tool, prompt style, and trust level.
• •
  PR volume has jumped and review is now the bottleneck.
• •
  Management is asking "what's our AI ROI" and you have no measurement story.

Skip the heavyweight version for a 2-person team — adopt just the provenance rule and the review gate.

Prerequisites

• •
  Authority (or sponsorship) to set team norms and touch CI/branch-protection config.
• •
  Your org's existing security/data policy for AI tools — you are specializing it, not replacing it.
• •
  Baseline delivery metrics for the last quarter: lead time, deploy frequency, change-failure rate, review turnaround. Without a baseline, you cannot claim anything later.

The workflow

1. 1.

   Sanction tools explicitly — including what data may flow to them. Publish a one-page table: tool, allowed plan (e.g. Copilot Business with no-training flag, not personal accounts), what may be pasted (code yes, customer data never, credentials never, client code per contract), and the escalation contact. Ambiguity here is how shadow-AI happens. Reference cases: Coinbase mandated tooling with mandatory human review; Shopify made usage an expectation — both paired the mandate with explicit guardrails.

2. 2.

   Set the provenance norm. Require PR descriptions to state AI involvement at the level that changes review behavior: AI-assisted (reviewed line-by-line) vs AI-generated (agent, human-verified via tests). This is not surveillance — it routes reviewer attention: provenance-aware review (pb from the review module) applies different checks to agent-generated diffs (hallucinated APIs, phantom tests, over-broad changes).

3. 3.

   Install the review gates.

   • •
     Author-side: AI pre-review pass (Copilot code review or equivalent) must run before requesting human review; nits die before humans see them.
   • •
     Encode team standards in copilot-instructions.md / review-bot config so the machine enforces conventions and humans review design.
   • •
     Hard rule: no AI-generated PR merges on green CI alone; a named human owns every merge. Write it into CODEOWNERS/branch protection, not just a wiki.
4. 4.

   Make the codebase agent-ready — this is the delivery unlock. DORA's amplifier capabilities are mostly platform work: fast deterministic test suite, linters and type checks as agent feedback loops, a maintained CLAUDE.md/AGENTS.md with build-run-test instructions and conventions. A team-owned instructions file beats ten private prompt libraries.

5. 5.

   Instrument outcomes, not activity. Track monthly, against your baseline: lead time for changes, review turnaround, change-failure rate/rollbacks, escaped defects, and PR size distribution (agent PRs trend large; large PRs kill review quality). Explicitly do not rank individuals by AI usage — acceptance-rate leaderboards optimize for accepting bad suggestions.

6. 6.

   Review the policy quarterly. Tools change monthly; a stale policy loses the room. Rotate a "tooling steward" who trials new capabilities in a sandbox repo and proposes amendments.

Tool variants

• GitHub-centric: Copilot Business/Enterprise + Copilot code review with instructions files + branch protection; provenance via PR template checkboxes.
• GitLab-centric: Duo + approval rules + .gitlab/ review config; same norms, different spelling.
• Mixed/BYO-agent teams: standardize on AGENTS.md (tool-agnostic) plus CI as the universal gate; allow tool choice, sanction data-handling centrally.

Verification

• Scenario testrun the policy against three real scenarios and check it gives a clear answer: an engineer wants to paste a stack trace containing customer emails; an overnight agent opens a 2,000-line PR; a contractor asks to use a personal ChatGPT account on client code.
• After 30 days≥90% of PRs carry provenance labels without chasing; review turnaround did not degrade.
• After 90 daysat least one delivery metric moved against baseline, and change-failure rate did not rise. If output rose but delivery is flat, diagnose the constraint (usually review or test speed) — that is the policy's next iteration.

Pitfalls

• Measuring AI activity instead of delivery. Suggestion-acceptance rates and "AI lines of code" are vanity metrics that invite gaming.
• Policy-by-prohibition. Teams route around bans within a week, into unsanctioned tools. Sanction a good path; make it the easiest path.
• One review standard for all provenance. Reviewing an agent's 1,500-line PR like a human's 150-line PR guarantees rubber-stamping.
• Ignoring the review bottleneck. +98% PRs merged means review capacity is the new constraint; if you change nothing else, change how review is staffed and scoped.
• Writing the policy alone. Draft with the team's actual power users — they know where the bodies are buried, and co-authorship is what makes norms stick.

Restricted-environment variant

Client-services context (the common Indian IT-services case): the client's contract overrides your policy. Maintain a per-engagement addendum: which tools the client sanctions (often Copilot Business under the client's tenant, or Bedrock/Azure OpenAI inside the client's cloud), redaction-first prompting rules for anything that leaves the client boundary, and a default-deny rule when the contract is silent — with a named person who can get an answer in writing. Local models (Ollama on approved hardware) cover comprehension and boilerplate where nothing may leave the machine.

Success criteria

• •
  A one-page policy every engineer can restate: sanctioned tools, data rules, provenance, review gates.
• •
  Provenance labels and AI pre-review are routine within a month — enforced by templates and CI, not nagging.
• •
  A quarterly metrics review that answers "is AI speed reaching delivery?" with data against a baseline.
• •
  Zero shadow-AI incidents, because the sanctioned path is genuinely the path of least resistance.