Design the Business-Side Governance Controls for a Deployed AI Process

Workflow area: AI Rollout & Governance · Tier: Advanced · Hands-on time: ~40 minutes

Deterministic systems stay tested; AI systems un-test themselves — the vendor updates the model, the input mix shifts, the prompt gets "improved," and last quarter's eval scores now describe a system that no longer exists. Engineering owns the monitoring stack, but the controls that keep a live AI process trustworthy are business processes: who reviews sampled outputs against what rubric, who may change what with what evidence, and who answers when a customer appeals. Designing those workflows is BA work, and this playbook is the operating manual you leave behind.

When to use — and when not to

• Use it before go-live of any AI process that mattersthe controls belong in the launch plan; retrofitting governance after the first incident costs ten times more and buys less trust.
• Use it when AI decisions can be challengedanything customer-affecting needs an appeal workflow with SLAs, and DPDP/GDPR grievance and automated-decision rights make one legally advisable.
• Use it when audits or certification loommapping each control to NIST AI RMF (or ISO 42001) converts your operating procedures into the evidence pack the auditor asks for.
• Skip the full apparatus for low-stakes internal toolsa meeting summarizer needs an owner, a feedback channel, and a quarterly look, not a review board; write the proportionality decision down and move on.
• Skip duplicating engineering's observabilitylatency dashboards and drift detectors are their controls; yours govern human decisions about the system. Reference, don't rebuild.
• Do not create governance without named ownersa control assigned to "the business" is assigned to nobody; if you cannot name the role, the control does not exist yet.

The recipe

1. 1.

   Stand up the sampled-output review as a standing workflow. Automated evals catch what the golden set anticipates; sampled human review catches what it doesn't. Specify it like any recurring process: sample size and method (random plus oversampled low-confidence and exception cases), cadence (weekly at launch, relaxing on evidence), the rubric (the golden set's, pb-48 — not fresh opinion), named reviewers with a calibration check between them, and the routing rule for findings — new failure patterns become golden-set candidates and, above a threshold, incidents (pb-52). Publish the trend line; a review whose results nobody sees will quietly stop happening.

2. 2.

   Design the exception and appeal workflows with SLAs. Two distinct flows, both first-class processes: the exception queue (system-initiated — low confidence, validation failure; largely designed in pb-46, now given operating ownership, staffing, and SLA reporting) and the appeal path (person-initiated — a customer or employee challenges an AI-shaped outcome). For appeals specify: how a person triggers review in plain language, who re-decides (a human with authority to override, seeing the full context per pb-45), the SLA, and the feedback rule — upheld appeals are system defects, so they route into the same triage as review findings. An appeal path that quietly re-runs the model is not review.

3. 3.

   Gate every change with regression evidence. The control that prevents silent decay:

   Copy

   MODEL-CHANGE GATE (applies to: model version, prompt, RAG
   source set, vendor platform update, threshold/config change)
   1. No change reaches production without a full eval run on
      the current golden set (GS-x, pinned version).
   2. Ship criteria: overall score ≥ baseline − [tolerance];
      zero-tolerance categories still zero; latency and cost
      within budget (pb-43 clauses re-verified).
   3. Sign-off: business owner + [role] approve on the evidence
      pack; approval recorded with eval report attached.
   4. Vendor silent updates: monthly scheduled eval run detects
      unannounced drift; a drop beyond tolerance triggers the
      incident process and vendor escalation per contract.
   5. Rollback: previous configuration restorable within [N]
      hours; rollback rehearsed [quarterly].
   

   The vendor clause matters most in SaaS reality: you cannot stop upstream updates, but a scheduled eval run means you find out from your own evidence, not from customers.

4. 4.

   Specify the audit trail and the review cadence. The audit-log content requirements exist from pb-45; governance adds the operating layer — who checks that logging still works (a quarterly reconstruction drill: pick a case, rebuild its decision chain, time it), where review results, gate approvals, and appeal outcomes are filed, and retention consistent with pb-47's privacy clauses. Then set the rhythm: a monthly operating review (owner, metrics, findings, actions) and a quarterly deep review that also re-asks the pb-51 question — is this still delivering? — with authority to order fixes or wind-down.

5. 5.

   Map controls to the framework and publish the one-pager. Tag each control with the NIST AI RMF function it serves (Govern / Map / Measure / Manage) or the ISO 42001 clause, producing the two-column evidence map auditors love. Then compress everything into a one-page operating model — controls, owners, cadences, escalation lines — because the governance that fits on one page is the governance that survives its author's departure. Hand it to the process owner; your name should not be load-bearing.

Make it yours

• Governance-lite tier: for medium-stakes tools — monthly 20-case sampled review, change gate with eval rerun, a feedback channel, one named owner; four lines that prevent most decay.
• Workflow tooling: run reviews, gates, and appeals in Jira or ServiceNow with templated tickets, so evidence accumulates as records instead of meeting minutes.
• Vendor-heavy variant: where the AI is bought, controls tilt contractual — update-notification clauses, eval-rerun rights, quality SLAs with remedies; the RFP claims table (pb-29) told you what was promised, the change gate verifies it quarterly.
• At portfolio scale: an AI register — every deployed system, its owner, risk tier, last eval date, last review — is the artifact that turns scattered controls into an auditable management system.

Quality check

• Every control has a named role, a cadence, and a filing location"someone should monitor this" appears nowhere in the document.
• Sampling oversamples the danger zoneslow-confidence, exception, and appeal cases are reviewed at higher rates than random flow.
• The change gate covers prompts and vendors, not just modelsprompt edits and silent SaaS updates pass through the same evidence requirement.
• Appeals reach a human with override authoritywith an SLA, full context, and upheld appeals feeding defect triage.
• The reconstruction drill is scheduledaudit-trail integrity is tested quarterly, not assumed from the logging spec.
• The framework map is two-wayeach control cites its RMF/ISO clause, and each applicable clause finds a control or a documented exclusion.