"The summary shall be accurate" is untestable — and requirements for non-deterministic systems need a new grammar: eval-referenced acceptance criteria, failure-mode catalogs, oversight clauses with legal deadlines. This module teaches that grammar, then the signature artifact: the process redesigned around an agent, with autonomy tiers, exception queues, adoption and measurement wired in from day one.
Priya is a Lead BA at a Chennai health-insurance company, ten years in, writing requirements for her first AI feature: automated summarization of claim files for adjudicators. She writes them the way she has written requirements for a decade — and they are, by classical standards, fine: "The system shall generate an accurate summary of the claim file. The summary shall include all relevant information. Summaries shall be produced in a timely manner." The vendor accepts them. UAT tests twenty hand-picked claims; the summaries read beautifully; every box ticks. Go-live.
Week five: an adjudicator denies a claim based on a summary stating the policyholder had a pre-existing-condition exclusion. The policy had no such exclusion — the model had conflated two similarly named conditions from different documents in the file. The denial triggers a complaint, a regulatory notice, and an internal review with an unanswerable question at its center: which requirement did the system violate? "Accurate" had no measure, no test set, no threshold. The requirement could not fail a test, so the system could not fail the requirement — while failing the patient. Nobody had specified what should happen when the model was uncertain, what the adjudicator should be told about AI involvement, how hallucinated clauses would be detected, or who reviewed what before a denial went out.
Priya's requirements weren't lazy; they were pre-AI. Deterministic systems let "shall be accurate" cash out into code review and fixed test cases. A probabilistic system needs the requirement itself to carry the measurement: the eval set, the threshold, the failure behavior, the oversight point. Writing that is a BA skill — the one this module teaches, and the one almost no BA curriculum covers.
For AI features, requirements and evaluation are the same artifact viewed from two sides. A classical spec references test cases; an AI spec references eval sets and rubrics. If your acceptance criteria don't say how quality is measured, on what cases, at what threshold, they aren't criteria — they're aspirations with section numbers.
The rewrite of Priya's spec, clause by clause. Instead of "the summary shall be accurate":
Two people must be unable to disagree on pass/fail — Module 3's testability gate, now load-bearing at production scale.
Classical specs assume failure is exceptional. AI specs assume failure is a rate — so every failure mode becomes a numbered requirement with five mandatory fields: detection signal (how the system or process knows it's happening), user-visible behavior (what the human sees instead of silence), fallback path (what happens to the work), human handoff (who, with what context), logging (what's recorded for audit and improvement).
| Failure mode | The requirement must specify |
|---|---|
| Hallucination / unfaithful output | Grounding-check or sampling-review detection; flag-don't-assert behavior; manual path; audit log of output vs source |
| Refusal / non-answer | Retry-then-route logic; user messaging; queue SLA for the manual fallback |
| Stale data | Maximum source-data age; behavior when exceeded; freshness surfaced to the reviewer |
| Prompt injection via input documents | Input handling rules; privilege limits (what the feature can never do regardless of content); detection logging |
| Latency spike | Timeout thresholds; degraded-mode behavior; user messaging beyond P95 |
| Cost runaway | Budget alarms and caps; throttle-vs-shutdown decision authority; notification chain |
AI helps draft the catalog — enumerate modes, propose detections — but each row is signed off by the analyst and its operational owner, because a fallback path that names no queue and no SLA is prose pretending to be a requirement.
Human-oversight requirements answer four questions concretely: where a human approves before consequence (in Priya's case: any denial-supporting summary is reviewed before adjudication completes); what the reviewer sees (the summary, source citations per claim-relevant statement, flags and confidence, the raw file one click away — oversight without context is rubber-stamping with extra steps); override mechanics (how the human corrects or discards, with corrections logged as future eval cases); audit trail (content and retention of the record: inputs, outputs, reviewer, decision).
Transparency now carries a legal deadline: EU AI Act Article 50 — in force since 2 August 2026 — requires, among other obligations, that users be informed when interacting with an AI system and that certain AI-generated content be disclosed. High-risk classifications under the Act carry Article 14 human-oversight duties with real design consequences. For any system touching EU users, the BA is who converts those legal texts into system requirements — disclosure wording, where it appears, what the audit log proves. India's DPDP Act and the DPDP Rules 2025 (notified November 2025, phased compliance through 13 May 2027, penalties to ₹250 crore) add consent-basis, purpose-limitation and notice obligations that AI features inherit the moment they process personal data.
"The system shall comply with applicable regulations" is the compliance version of "shall be accurate" — untestable, unownable, and useless in an audit. Compliance arrives as specific, testable clauses: this disclosure text, at this point in this flow; this consent basis, verified this way; this log, retained this long. Writing those clauses is now billable BA work with a deadline attached.
Fairness and explainability follow the same grammar: fairness checks per affected segment with counterfactual eval cases attached (same claim, demographic attributes varied — outcomes must not materially differ; measured, not asserted), and an explainability level per decision type ("the adjudicator can see which source document supports each summary statement") specified as behavior, not virtue.
Module 7 decided where agents belong; this is how the process is rebuilt around one — in BPMN, walkable with SMEs like any process, because an agent process is still a process. Three design moves:
Autonomy tiers per step. Not one autonomy decision for "the agent," but one per step:
The agent drafts and recommends; a human performs the action. Every new agent step starts here, generating the evidence — quality rates, correction rates — that could justify promotion.
The agent prepares the action fully; execution follows an explicit human approval with adequate context. The reviewer's queue has an SLA — an unstaffed approval queue is where agent processes silently die.
The agent executes within hard bounds — amount limits, category limits, rate limits — logging everything for sampled review. Reserved for reversible, low-blast-radius actions with proven track records.
Exceptions as first-class branches. The defining BA contribution: exception queues, escalation paths and confidence-based routing drawn as primary process branches with owners and SLAs — not an "agent handles it" happy path with error handling to-be-determined. Where does a below-threshold case go? Who owns the exception queue, staffed at what level, at what SLA? When does repeated failure escalate from case-level to process-level (pause the agent)? An agent process whose exception paths are afterthoughts is Module 4's merged-exception-path failure, rebuilt at machine speed.
SLAs on the human steps. Tier-2 review time is now on the critical path; specify it, staff it, measure it — or the business case's cycle-time benefit quietly evaporates in an approval queue.
The golden dataset threads through everything and is BA-owned: curated from the same interviews, tickets and exceptions you elicited in Module 2 — the real cases, including the ugly ones — with expected behavior and pass criteria per case, versioned like any baselined artifact. It is referenced by the acceptance criteria, reused for vendor evals (Module 7) and regression gates, and grown continuously from production corrections and incidents. Requirements skills are dataset-curation skills: knowing which cases represent the business, which edge cases matter, and what "correct" means per case is precisely elicitation-and-specification competence pointed at a new artifact.
Change impact, AI-flavored. From the to-be map: whose tasks change how, task by task. Address trust in both failure directions — under-trust (reviewers redoing everything, deleting the benefit) and over-trust (rubber-stamping, deleting the safety). Every job aid states verification duties explicitly: you own what you approve — the Tier-2 reviewer is accountable for approved actions, and their job description, training and workload must say so. This is where McKinsey's adoption evidence bites: AI ROI lives or dies in whether changed roles actually work, which makes change analysis a load-bearing artifact, not a communications afterthought.
Governance as a runnable process. Sampled-output review on a cadence with named reviewers; a model-change gate — no prompt or model version ships without golden-set regression evidence (vendor "upgrades" silently change behavior; your gate is the only defense you control); exception and appeal workflow for affected customers; incident classes defined business-side — mass wrong answers, discovered bias, injection exploit, cost runaway — each with a severity ladder, a named shutdown authority (someone must be empowered to stop the agent today), remediation templates, and regulator triggers including DPDP's 72-hour breach notification. Documented as process and RACI, mappable to NIST AI RMF for the auditors.
Honest measurement. The Module 7 charter, executed: baseline already captured; leading indicators (usage, containment, override rates) tied to lagging outcomes (cost, cycle time, error rates); novelty effects and displacement costs accounted; and a quarterly benefits-realization review allowed to conclude "not working — kill or fix." A measurement process that can only report success is a marketing process. The analyst who runs one that can say no is the reason the next initiative gets funded.
Walk the agent-era BPMN with SMEs exactly as in Module 4 — same script, plus tier-specific questions at every agent step: "what do you see before approving? what if you're unsure? what if it's wrong twice in a row — who do you tell, and what stops?" If the map can't answer an operator's question, the process isn't designed yet — no matter how good the model is.
Non-deterministic systems need requirements that carry their own measurement: eval-referenced criteria, failure modes as numbered clauses, oversight with legal deadlines, autonomy earned tier by tier, exceptions as first-class branches, and a golden dataset the BA owns. Requirements, evaluation and governance have merged into one discipline — and the analyst who writes requirements that can actually fail a test is the one who keeps AI systems honest.