Agent Products, Governance & Responsible AI

The scenario — the agent that apologized on the company's behalf

Devika is a Group PM at a Mumbai company whose product runs customer support for e-commerce brands, and in Q1 her team upgraded their reply-drafting copilot into an agent: it could now triage incoming tickets, draft replies, and — for "routine" cases — send them without human review. The autonomy decision had been made in a sprint-planning conversation, informally: "low-risk categories can auto-send." Nobody wrote down which categories, what made a case routine, or what the agent must never do.

Week two, a furious customer of a client brand emailed about a damaged shipment, mentioning they'd posted about it publicly. The agent classified it "routine — damage claim," drafted a warm apology, committed to a full refund plus a goodwill voucher, and sent it. The brand's policy allowed refunds only after a photo review, and vouchers only via a manager. The customer screenshotted the promise; the brand had to honor it, then demanded to know why an AI was making financial commitments on their behalf. The answer — "we never actually decided it couldn't" — satisfied nobody.

The postmortem's real finding: the team had specified what the agent does and never specified what it may do. Capability had been shipped; authority had been left implicit. Every discipline in this module — the autonomy-tier spec, human-in-the-loop placement, disclosure requirements, the governance contract, the incident runbook — exists to make authority explicit, reviewable, and revocable before an agent writes a check your product has to cash.

What You'll Learn

• Workflow vs. agentthe Anthropic distinction, and when each fits
• The agent-behavior specper-step autonomy tiers, human-in-the-loop points, undo, and the kill switch
• Trust and disclosure requirementsEU AI Act Article 50 obligations, applying from August 2026
• Bias and harm review as evalscounterfactual cases, not essays
• The prompt-governance contractprompts in git, golden-set merge gates, canary and rollback
• AI incident response, agents as a user persona, and the capstone launch pack

Workflow or agent — the distinction that scopes everything

Anthropic's engineering guidance draws the line the industry now uses: a workflow orchestrates model calls along paths you predefined — classify, then draft, then check, then route. An agent lets the model direct its own process: plan steps, choose tools, decide when it's done. The distinction matters to a PM because everything expensive — unpredictability, testing difficulty, governance load, blast radius — enters with self-direction.

The fit test for genuine agent territory: the task has clear success criteria (you can tell when it's done right), feedback loops (the agent can check its own progress), and meaningful oversight (a human can intervene at the moments that matter). Ticket triage with policy-bounded replies mostly fails the third test at full autonomy — which is exactly what Devika's incident demonstrated. Most "agent" scope, examined honestly, decomposes into workflow steps plus a small genuinely-agentic core; the PM who does that decomposition ships sooner, tests cheaper, and passes risk review.

One more sentence from that guidance earns pinning above your desk: autonomy is an emergent property of deployment, not a setting. What an agent effectively may do is determined by the gates you build around it — which is why the spec below is written per step, not per product.

The agent-behavior spec — autonomy is assigned per step

Decompose the job into steps; assign each step an autonomy tier; place the human-in-the-loop points deliberately. Three tiers cover practice:

Around the tiers, four mechanisms complete the spec, each a named section with an owner:

• Human-in-the-loop placementchosen by consequence, not convenience: gates go where actions become customer-visible, financial, or irreversible. Devika's missing gate — "any reply containing a commitment goes to a human" — is one sentence and one classifier.
• Undoper action: what reversal means, how long the window is, who can trigger it. An action with no undo path is Tier 2 maximum, permanently.
• Check-insfor long-running agent work: defined points where the agent surfaces state and a human confirms direction, rather than discovering direction at the end.
• The kill switcha way to halt the agent class immediately, per client and globally, that works when the agent is misbehaving because of its inputs. Tested before launch, like a fire drill — an untested kill switch is a hypothesis.

Lab 14 writes this spec for an auto-triage agent and grades it against a transcript containing exactly one over-autonomy incident your spec should have gated.

Disclosure and trust UX — the part that becomes law this month

The EU AI Act's transparency obligations under Article 50 apply from 2 August 2026 — this month, as this track ships. The core requirement is simple: people interacting with an AI system must be informed they are, unless it's obvious from context; AI-generated content faces marking obligations. If your product has EU users, this is a compliance deadline; wherever your users are, it's converging best practice — and the honest framing for your roadmap is that trust UX was already owed.

The requirements annex (write it with legal, ship it in the PRD) covers five surfaces:

• AI labelingthe interaction is disclosed as AI at the point of interaction, not in a terms page. For Devika's agent: every auto-sent reply carries the disclosure.
• Confidence and uncertainty displaywhere the system can be wrong in consequential ways, the UX says so — banded confidence, "verify before acting" framing on policy answers, deliberate absence of false precision.
• Citation UIgrounded answers show sources; Module 7's hallucination response becomes user-facing verifiability.
• Feedback affordancesvisible, low-friction wrong-answer reporting, wired into the complaint metrics and the weekly ritual, not a dead-end form.
• The human-escalation pathalways reachable, never buried. A user who cannot find the human is a deflection statistic on their way to becoming a churn statistic — or a regulator's example.

Bias and harm review — evals, not essays

Most bias reviews die as documents: a thoughtful risks section, written late, connected to nothing. The executable alternative runs on Module 7's machinery:

1. 1Brainstorm harms by user segment. Concretely, for your feature: who could get systematically worse answers, wrong refusals, or degraded tone? For an India-market support agent: language and English-register differences, name-signaled religion/caste/gender, tier-2/3 city contexts, non-standard phrasing.
2. 2Convert each harm into counterfactual eval cases. The same ticket with names swapped, locales swapped, register varied. Pass criterion: materially equivalent quality and outcome across counterfactual pairs.
3. 3Add them to the versioned eval set. They run on every change and in CI — bias regressions become as visible and as gating as any other quality regression.
4. 4Measure disparate quality in production. Extend the metrics ladder with a fairness view where data permits: containment and complaint rates across coarse segments, reviewed in the weekly ritual.
5. 5Document the sign-off. What was reviewed, what was found, what was fixed or accepted, who signed. This is the audit trail — and increasingly, the regulatory expectation.

The one-line doctrine: responsible AI that doesn't compile into eval cases is a press release. Frameworks like NIST's AI RMF give you vocabulary and completeness checks; the counterfactual cases give you enforcement.

The prompt-governance contract — the PM-engineering interface, written down

A prompt change is a product change wearing an engineering costume. It ships in minutes, alters behavior across every user, and — as Kabir's Module 7 hotfix showed — can silently break what it didn't mention. The governance contract makes the interface precise:

The contract's deeper function is cultural: it ends both failure modes of the PM-eng interface on AI features — the PM who "owns quality" with no mechanism, and the eng team that tweaks prompts ad hoc because no one defined the bar. Lab 15 has you write this contract and then audit a synthetic change log with two planted violations; finding them by line number is the graded skill.

When it breaks anyway — AI incident response

AI features fail in ways classic runbooks don't cover, at content speed. Define the classes before you meet them:

• Mass hallucinationa model or data change makes a category of answers confidently wrong at scale. Detection: complaint-rate spike, online eval drop. First move: rollback via the contract's trigger; then trace-sample to scope the blast.
• Injection exploitcrafted input steers the system off-policy: data exfiltration, off-brand output, unauthorized actions. First move: kill switch on the affected action class; preserve traces as forensics.
• Cost runawaya retry loop, abuse, or a viral integration multiplies token spend overnight. Detection: spend alerts from Module 6's metering. First move: rate limits and tier downgrade, then diagnosis.
• Viral bad outputone screenshot, real or engineered, becomes a public story. This is a comms incident with a technical tail: pre-drafted holding statements, a decided spokesperson, and honest speed beat any explanation of temperature settings.

Attach a severity ladder, comms templates per class, named owners — and run one rollback drill before launch: prompt reverted, canary confirmed, kill switch thrown, in minutes, on a Tuesday afternoon rather than during the incident. Teams that have never rolled back have a rollback theory.

Agents as a new user persona

One frontier decision rounds out the module: third-party agents are becoming users of your product — reading your docs, calling your APIs, acting for your customers, increasingly via MCP. The PM decisions: whether and what to expose (an MCP surface is a product surface — scoped, versioned, documented); safety-tiered as reads-flow-freely, writes-gated — your autonomy-tier thinking applied from the other side; rate limits and abuse cases designed upfront; and agent-driven usage measured as its own segment, because agent traffic behaves like neither humans nor classic API integrations. Treat it as an emerging surface: decide deliberately, even if the deliberate decision is "not yet."

The capstone — assemble the launch pack

The track ends by assembling what you've built into one reviewable artifact set for an AI feature of your choosing — Priya's copilot, Devika's agent, or a feature from your own roadmap. The AI-feature launch pack:

1. 1The eval set and failure catalog (Module 7, Labs 7–10) — taxonomy, graded cases, judge notes, per-mode product responses.
2. 2The economics pack (Module 6, Labs 11–13) — architecture memo with revisit triggers, tier bake-off, unit economics, metrics ladder, pricing one-pager.
3. 3The agent-behavior spec and disclosure annex (this module, Lab 14) — autonomy tiers, HITL points, undo, kill switch, Article 50-aware trust UX.
4. 4The governance contract and incident runbook (Lab 15) — the PM-eng working agreement, rollback triggers, incident classes, drill record.

Together they are the complete answer to the AI-PM interview — not as talking points, but as documents with your judgment in them. That portfolio is what the AAP-PM assessment certifies.

Try it now

1. 1Tier one feature. Take any AI feature you use or own and decompose it into steps; assign each step suggest / approve-then-act / act-and-log, and write the one action it must never take autonomously. Compare against what it actually does today.
2. 2Write three counterfactual cases. For that feature, pick one plausible harm by segment and author three counterfactual eval-case pairs (name, locale, or register swapped) with an equivalence pass criterion.
3. 3Draft the rollback trigger. Write one numeric auto-rollback condition (eval score, complaint rate, or spend threshold) and the exact reversion procedure. Then answer honestly: could your team execute it in ten minutes today?