Write Probabilistic Requirements for an AI Feature

Workflow area: Evals & AI Quality · Tier: Advanced · Hands-on time: ~45 minutes

Deterministic software either meets its spec or has a bug. An LLM feature meets its spec 94% of the time, and the whole product question is: which 94%, how do you know, and what happens to the users in the other 6%? Classic PRD language cannot express any of that, which is why AI features shipped against classic PRDs generate the same fight every time — engineering says "it works," support says "it's wrong all the time," and nobody defined what either claim means. Probabilistic requirements end that fight before it starts.

When to use — and when not to

• Use it for any feature where an LLM output reaches userschat, summarization, generation, classification with user-visible consequences; if the output can be wrong without a bug existing, the feature needs this annex.
• Use it when engineering asks "how good does it have to be?"that question, asked seriously, is the request for this document.
• Use it before pb-37's eval set is commissionedrequirements define what the eval set measures; write the bars, then build the instrument.
• Skip the full annex for internal, human-reviewed drafting toolswhere a human edits every output before it matters, a lighter quality note suffices; spend the rigor where outputs act on the world.
• Skip inventing thresholds you can't yet measurea bar with no eval behind it is theater; each threshold in the annex must name the eval that will measure it, even if that eval doesn't exist yet.

The recipe

1. 1.

   Define the quality dimensions before the numbers. For your feature, name the 3–5 dimensions that matter — factual accuracy against the knowledge source, instruction-following, tone/brand safety, refusal correctness (refusing what it should, not refusing what it shouldn't), citation validity. Generic "accuracy" hides the trade-offs; a support copilot can trade a point of fluency for a point of policy-correctness, and the annex should say which way.

2. 2.

   Draft the annex with AI doing structure and you doing the bars.

   Copy

   Draft a probabilistic-requirements annex for this AI feature.
   Structure it as:
   
   1. QUALITY BARS — a table: dimension | metric | how measured
      (which eval) | launch bar | aspiration. Leave the numeric
      bars as [PM TO SET] — do not invent thresholds.
   2. FAILURE-MODE TABLE — for each plausible failure mode:
      user impact, expected frequency band, severity (S1-S3),
      detection signal, product response.
   3. FALLBACK & DEGRADATION UX — what the user sees on: low
      confidence, no retrieval hits, timeout, refusal, provider
      outage. Each state needs UI copy intent and an escape hatch
      to a non-AI path.
   4. LATENCY & COST BUDGETS — p50/p95 latency targets, cost
      ceiling per interaction at projected volume.
   5. NON-REQUIREMENTS — what this feature explicitly does NOT
      promise in v1.
   
   Feature description: [PASTE]
   Knowledge source & volatility: [PASTE]
   User and stakes: [who relies on this, cost of a wrong answer]
   

3. 3.

   Set the bars yourself, anchored to consequences. For each dimension ask: what does a miss cost? A wrong refund-policy answer costs a refund and a trust dent — maybe 98% policy-accuracy launch bar with S1 classification for confident errors. A blandly-toned answer costs little — 90% is fine. Write the reasoning next to each number; a bar whose rationale is recorded can be renegotiated honestly later, and interviews for AI-PM roles probe exactly this reasoning.

4. 4.

   Specify behavior at the boundary, not just the average. The annex's most valuable rows describe the bad tail: what the product does when the model is unsure (show sources and hedge, or decline?), when retrieval returns nothing (say so — never let the model freestyle), when the same user hits three bad answers in a session (offer the human handoff proactively). Every degradation state gets an owner and UI copy intent — this is where pb-39's catalog plugs in.

5. 5.

   Make the requirements testable line-by-line. Walk the finished annex with this filter: for each requirement, could an eval or a monitor return pass/fail against it? "Feels trustworthy" fails the filter; "≥95% of answers containing a factual claim include a citation that resolves to a source document" passes. Rewrite until every line is measurable, and name the eval (pb-37) or the production metric (pb-41) that will measure it.

6. 6.

   Review with engineering and design as a contract. Engineering confirms the bars are measurable in their pipeline and the latency/cost budgets are physical; design owns the degradation states as real screens, not error toasts. The annex then becomes the reference document for pb-40's launch gates — ship/no-ship is evaluated against these numbers, not against vibes in the launch review.

Make it yours

• ChatPRD/template route: encode the five-section annex as a template in your PRD tool so every AI feature PRD gets the sections scaffolded; empty sections are visible in review, which is the point.
• The retrofit: run the annex exercise on an AI feature you already shipped — filling in the failure table from live tickets is sobering and produces pb-36's trace-pull shortlist for free.
• Stakes-tiered rigor: maintain two versions — the full annex for user-facing or action-taking features, a one-page "quality note" (bars, top three failure modes, one fallback) for low-stakes internal tools — so rigor scales with consequence instead of being rationed politically.

Quality check

• Every quality bar names its measuring instrumentthe eval, judge or production metric that returns a number against it.
• Bars carry rationale anchored to consequenceeach number has a because-sentence, not a vibe.
• The failure-mode table covers the top modes with severity and responsehallucination, staleness, wrong refusal at minimum, per your feature.
• Degradation states are specified as UX, with escape hatcheslow confidence, empty retrieval, timeout and outage all have defined user experiences and a path to a human or non-AI alternative.
• Latency and cost budgets exist and were sanity-checked with engineering at projected volume.
• Non-requirements are explicitthe annex says what v1 does not promise, which is what protects the team from scope-by-anecdote after launch.