Turn the AI Policy into CI Gates That Enforce Themselves

Workflow area: Policy & Governance · Tier: Advanced · Hands-on time: ~45 minutes

Every AI policy fails the same way: it is announced, agreed with, and then outrun by a Tuesday deadline. Nobody pastes a secret into a prompt out of malice — they do it at 6 p.m. with a demo tomorrow. Enforcement beats goodwill not because your engineers are untrustworthy but because pipelines do not get tired, and a gate that catches the mistake before merge protects the engineer as much as the codebase.

When to use — and when not to

• Use it once a written policy existsthis playbook enforces rules; pb-27 and your policy doc define them. Gating rules nobody agreed to is how you manufacture resentment.
• Use it when AI-assisted volume makes manual vigilance a fictionpast a handful of assisted PRs a day, "reviewers will catch it" is a hope, not a control.
• Use it after your first near-missa hallucinated package that almost shipped, a key that almost merged. Near-misses are your cheapest political capital for gates.
• Skip blocking mode on day onea gate that breaks builds before the team trusts its false-positive rate gets disabled within a week, and re-enabling a disabled gate is ten times harder than staging it right.
• Skip gates you cannot staffevery gate needs an owner for its alerts and exceptions; an unowned gate becomes either noise or an outage.

The recipe

1. 1.

   Extract the checkable rules from the policy. Run the policy doc through Claude:

   Copy

   Here is our engineering AI-use policy: [PASTE POLICY].
   
   Produce a table with columns: policy rule / can CI check it?
   (YES, PARTIAL, NO) / concrete check / suggested tool class /
   failure action (warn or block).
   
   Be strict: "developers should review AI output carefully" is NOT
   checkable — say NO. Secrets in diffs, license headers, unknown
   dependencies, coverage drops, missing provenance labels ARE.
   Do not invent tool names beyond well-known classes.
   

   Good output separates the enforceable from the aspirational:

   Copy

   No secrets in prompts/commits | PARTIAL | diff secret scan   | gitleaks-class    | block
   No copyleft contamination     | YES     | license scan       | ScanCode/FOSSA    | warn→block
   Deps must exist + be mature   | YES     | registry age check | custom CI step    | block
   Coverage floor on ELEVATED    | YES     | coverage delta     | coverage gate     | block
   "Review AI output carefully"  | NO      | —                  | culture, not CI   | —
   

2. 2.

   Stand up the four core gates. Secrets scanning (Gitleaks/TruffleHog-class) on every push — this one starts in block mode; a leaked key is not a teachable moment. License and provenance scanning (ScanCode/FOSSA-class) on new and changed dependencies, tuned to your license allowlist from pb-34. A dependency-hallucination check: every newly added package must exist in the registry, exceed an age threshold (e.g., 90 days), and clear a download floor — this is your defense against slopsquatting, where attackers pre-register the plausible package names models invent. Coverage floors on the risk tiers defined in pb-27.

3. 3.

   Roll out warn-first, then ratchet. Two weeks in warn mode per gate (except secrets). Each week, triage the hits: true positives become your announcement material ("the gate caught three real issues before merge"); false positives get tuned out. Announce the block date in advance, in writing. Flipping a gate to block without a warn period is how gates die.

4. 4.

   Build the exception path before the first exception. A policy-exception PR label plus a one-line justification, auto-logged to a review queue that you or a delegate clears weekly. No exception path means engineers bypass gates in the dark; a heavyweight one means the same thing with extra resentment. One label, one line, one weekly review.

5. 5.

   Report gate health monthly. Hits, true-positive rate, exception count, and time-to-fix — at team aggregate. A gate with a 40% false-positive rate is a tax; fix it or kill it. A gate with zero hits for a quarter either finished its job (keep it, it is cheap) or is miswired (test it with a deliberate canary commit).

Make it yours

• Small team or startup: Gitleaks in pre-commit plus the dependency-existence check gets you 80% of the protection in an afternoon. Skip the license-scanning SaaS until a customer or investor asks; do not skip secrets.
• Large org or GCC: run gates as a central paved-road CI template that teams inherit rather than copy, so tuning propagates; map each gate to the specific client or compliance clause it satisfies (pb-33) — auditors accept pipeline evidence far more readily than policy PDFs.
• Open-source-heavy codebase: the license gate is your headline act — enforce the allowlist on transitive dependencies too, and route AI-generated code that matches known OSS verbatim (provenance scan) straight to the pb-34 legal gate.

Quality check

• Every gate traces to a written policy linea gate nobody can source to the policy is a personal preference wearing a uniform, and it will be litigated forever.
• True-positive rate stays above roughly 60%below that, engineers learn the gate cries wolf and start clicking through; tune or demote to warn.
• Exceptions are rare and reviewedmore than a handful a week means the policy is wrong, the gate is wrong, or the team was never on board; find out which before enforcing harder.
• Canary test quarterlycommit a fake secret and a fake package on a branch and confirm both gates fire; a silent gate is worse than none because you are trusting it.
• Merge time did not degradegates that add minutes to every build get disabled by popular demand; keep them incremental and diff-scoped.