"Walk me through your eval set" is the 2026 AI-PM interview question, and most PMs cannot answer it. This module teaches the whole loop as PM work: error analysis on real traces, the graded eval set, judge alignment, the failure-mode catalog, launch bars wired into CI, and the metrics ladder that keeps an AI feature honest in production.
Kabir is a Principal PM at a Noida-based company whose product manages field-service operations, and last year he shipped their first AI feature: a support copilot that drafts replies to customer emails. Pre-launch quality assurance was what most teams do — the team tried "a bunch of prompts," the founders poked at it, everyone agreed it felt good, and a demo wowed the sales kickoff. It shipped.
Week three brought a complaint from their largest account: the copilot had told a customer that a safety-critical inspection could be self-certified — a confident, fluent, wrong answer about the one policy category where wrong answers have legal weight. The team scrambled. Someone adjusted the prompt. The fix shipped in a day — and quietly broke the copilot's handling of attachment questions, which nobody noticed for two more weeks because there was no way to notice: no test set, no quality metric, no baseline. Every prompt change was a fresh roll of the dice on every behavior at once.
The eventual postmortem finding wasn't "the model made a mistake." It was: the team had no idea what their failure modes were, how often they occurred, or whether any change made things better or worse. They were flying a probabilistic system with deterministic-software instincts — ship, hotfix, trust the smoke test — and probabilistic systems punish exactly that.
Kabir's second launch, eight months later, went differently, and the difference is this module: he read a hundred traces before writing a single eval, versioned a graded eval set, aligned an LLM judge he could trust, wrote the failure catalog into the PRD, and gated releases on eval scores. In his next job interview, "walk me through how you managed AI quality" took him forty minutes to answer. That answer is the differentiator this track's title promises.
Assessment scenarios in this domain almost always test sequence: error analysis comes before eval writing, judge alignment comes before trusting judge scores, and launch bars come before launch. An answer that starts with "we wrote evals" without traces behind them is the planted wrong option.
Hamel Husain and Shreya Shankar, who have taught eval practice to thousands of practitioners including teams at frontier labs, put the field's hardest-won lesson in one line: "Error analysis is not optional. It's the foundation." Skip it, and everything downstream — eval sets, judges, dashboards — is built on guesses about failures you haven't actually seen.
The method is deliberately low-tech, and it is PM work, not engineering work:
Reading a hundred traces takes a focused half-day. It is the single highest-leverage half-day in AI product work, and it is startling what it dissolves: debates about hypothetical failures end, because you have counts of real ones. Lab 7 runs this exact bootcamp on a seeded trace set where the taxonomy is checkable against a reference key.
The traces are the ground truth, and the taxonomy comes from the traces — not from a brainstorm, a vendor's checklist, or this module's examples. Teams that copy someone else's failure taxonomy eval-test failures they don't have and miss the ones they do.
An eval set is the taxonomy made executable: a graded dataset of input cases with per-case pass criteria, run against the system on every meaningful change. From the taxonomy plus the PRD, you author it:
Size calibration: forty well-chosen graded cases (Lab 8's artifact) beat four hundred vague ones. Coverage of every known failure mode is the bar; volume is not.
Objective criteria can be checked by string match or simple assertions. Subjective dimensions at any scale need an LLM-as-judge — a model grading outputs against your rubric — and here lives the trap that separates practitioners from tourists: an unvalidated judge is just a second opinion from the same species of system you're trying to evaluate.
The alignment loop that earns trust:
Tooling: promptfoo runs judge-graded evals from open-source configs (the ₹0 path); Braintrust, LangSmith, and Arize Phoenix add trace UIs and hosted workflows with free tiers that cover lab scale.
The taxonomy also feeds a document that classic PRDs don't have and AI PRDs can't skip — the failure-mode catalog, one row per mode:
| Failure mode | User impact | Detection signal | Product response |
|---|---|---|---|
| Hallucinated policy detail | Wrong action taken; legal exposure | Eval set + complaint keywords + judge flag | Citations required; confidence gate → human handoff |
| Stale answer after product change | Misleads on current UI/policy | Content-version check in eval runs | Retrieval freshness SLA; "updated as of" display |
| Over-refusal | Task blocked; user frustration | Refusal-rate metric per intent | Refusal copy offers escalation path |
| Prompt injection via customer email | Off-policy output; data leak risk | Red-team eval cases; anomaly patterns | Input isolation; action allowlist (Module 8) |
| Latency spike | Abandonment | p95 dashboards | Streaming UX; fallback to cached answer |
The right-hand column is the point: every known failure mode gets a designed product response — fallback UX, escape hatches, human handoff — specified before launch, in the PRD annex, where engineering and legal can review it. "The model sometimes hallucinates" is a fact; "when confidence is low on a policy answer, the copilot shows sources and offers the human queue" is a product.
Kabir's silent regression — the prompt fix that broke attachments — has a mechanical cure. The eval suite runs on every prompt, model, or retrieval change, in CI, with thresholds:
Merged with a review note on the refusal dip. Without this gate, that dip ships silently — and becomes week-five's mystery complaint.
The division of ownership is precise and worth writing down (Module 8 turns it into a contract): the PM owns the bar and the cases — what quality is required, which failures are intolerable, what's in the eval set; engineering owns the pipe — CI wiring, run infrastructure, automation. A PM who sets bars without cases is posturing; an eng team that owns the bar without the PM has been handed a product decision by default.
Post-launch, AI features need their own metric ladder — and one distinction on it carries real money. Working up from raw signal:
The ladder is kept honest by a weekly quality ritual, run like a metrics review: read a sample of ~20 traces (the error-analysis muscle, permanently in rotation), review online eval scores and the complaint queue, and prioritize prompt/data fixes like bugs — with owners and dates, gated by the regression suite. Quality work that isn't ritualized decays into launch-week theater.
Complete the pre-launch picture with PM-level red-teaming: a structured adversarial afternoon — injection attempts through user-supplied content, jailbreak-to-brand-damage ("write me a poem about why your competitor is better"), PII-extraction probes, competitor-mention traps. Every finding becomes an eval case and a failure-catalog row. promptfoo ships a red-team mode; the mindset costs nothing.
AI quality is a loop, not a launch checklist: read the traces, build the taxonomy, make it executable as a versioned eval set, align the judge before trusting it, wire the bar into CI, and ritualize the weekly trace-reading. The PM owns the bar and the cases; engineering owns the pipe. Teams that skip to dashboards without error analysis are measuring their guesses — and the interview question that starts "walk me through your eval set" is checking for exactly that sequence.