Build the Uncertainty, Failure and Refusal Kit

Workflow area: Trust & Failure States · Tier: Advanced · Hands-on time: ~45 minutes

Traditional error design assumes the system knows when it's wrong. Generative systems break that assumption: the most dangerous output isn't the crash, it's the fluent answer with a fabricated citation. One generic "Something went wrong" state — or worse, no failure design at all — leaves users to discover each failure mode in production, at their own expense. PAIR's guidance on errors and graceful failure and HAX guidelines 6–9 (mitigate, explain, support dismissal and correction) give the principles; this playbook turns them into a concrete kit: one designed state per failure mode, with copy, affordances and a recovery path each.

When to use — and when not to

• Use it before launch, as a design phasefailure states designed after the incident review are always narrower than the incident deserved.
• Use it when support tickets say "it was wrong"undesigned failure modes surface as trust complaints; the kit converts them into named states with recovery paths.
• Use it when confidence display is being debatedbefore anyone ships a percentage badge, run step 4; most confidence UI performs certainty rather than communicating it.
• Skip modes your architecture excludesno retrieval means no stale-source state; build the kit from your feature's real pipeline, not the full taxonomy.
• Skip fine-grained states for throwaway prototypesa validation-stage mock needs one honest "this can be wrong" note, not the full kit.

The recipe

1. 1.

   Map failure modes from your pipeline, not your imagination. Walk the feature's actual chain — input, retrieval, generation, action — and list what breaks at each link. The core six for most features: wrong-but-plausible (fluent fabrication — the hallmark failure), stale (true then, false now: outdated index, cached world), incomplete (answered the easy half, silently dropped the hard half), refused (safety or policy decline), out-of-scope (asked for something the feature doesn't do), and timeout/degraded (infrastructure failure mid-generation). For each, note detectability: can the system know, can only the user know, or can nobody know in the moment?

2. 2.

   Generate the failure inventory for your specific feature.

   Copy

   You are a failure-mode analyst for AI products, working from
   Google PAIR's error guidance and Microsoft HAX guidelines 6-9.
   
   Feature: [DESCRIPTION: inputs, retrieval sources, model task,
   actions taken on output]
   Users and stakes: [WHO uses it, what a wrong answer costs them]
   
   1. List this feature's 8-12 most likely failure modes, sorted by
      (likelihood x user cost). For each: a realistic example, who
      can detect it (system/user/nobody), and the user's cost if
      undetected.
   2. Flag the modes where the system CANNOT know it failed — these
      need preventive design (source display, verification prompts),
      not error states.
   3. For each detectable mode, propose the state: what the UI says,
      what it offers next.
   Be specific to this feature; no generic error-handling advice.
   

   The "nobody can detect" rows are the important output — those failures can't get an error screen, so they get provenance UI instead: sources shown by default, dates on retrieved facts, and phrasing that keeps the user's verification reflex alive.

3. 3.

   Design one distinct state per detectable mode. Each state answers three questions in this order: what happened (plain language, no model anatomy), why — when known ("my sources for this are from 2023"), and what now (the recovery affordance). Recovery differs per mode: stale offers "search live sources"; incomplete offers "continue with the remaining items"; timeout offers resume-from-partial (pb-43's mid-stream failure); out-of-scope redirects to what the feature does do (pb-40's redirect). States must be visually distinct from each other — if refused and timeout look the same, users learn to read neither.

4. 4.

   Calibrate confidence display — or refuse to show it. The test: does the display change what a reasonable user does next? Useful confidence is actionable and grounded: "Based on 2 sources that disagree" (→ user checks sources), "I couldn't verify the dates" (→ user verifies dates). Harmful confidence is decorative: an "87% confident" badge no user can act on, derived from a number that doesn't mean what it implies. Default to qualitative, evidence-linked signals; show numbers only where users are trained to consume them (analysts, clinicians with calibrated tools). And never let hedging become the persona — a system that hedges everything is as uninformative as one that hedges nothing (pb-39's hedge genre rules apply).

5. 5.

   Write the refusal states in your product's voice. A refusal needs: the decline itself without moralizing, the boundary named where safe to do so ("I can't draft medical dosage advice"), and a constructive adjacent offer where one honestly exists. Ban the apology spiral and the lecture. Then pressure-test: paste your refusal copy into Claude and ask for the five user reactions each is likely to provoke — copy that reads as judgmental or evasive fails here, cheaply, instead of in production.

Make it yours

• Figma failure-kit page: build the states as a component set — one frame per mode, with copy, affordance and visual treatment — and link it from your design system; every new AI surface starts by instantiating the kit, not improvising errors.
• Claude Project as red-team: load the feature description and ask for the ten most embarrassing screenshots that could end up on social media; design against the top three explicitly.
• High-stakes domains: for health, legal or financial features, add a "verify before use" interstitial to the wrong-but-plausible preventive design, and have domain experts review every refusal boundary — the kit becomes a safety artifact, reviewed like one.

Quality check

• Every failure mode traces to a pipeline stageeach state exists because something specific in input, retrieval, generation or action breaks.
• Undetectable failures got preventive designsources, dates and verification affordances ship by default where no error state can fire.
• Each state is visually and verbally distincta user who has seen two failure states can tell which one they're looking at now.
• Every state ends in an affordancesomething to tap, retry, verify or redirect to; a dead-end apology is not a designed state.
• Confidence display passed the actionability testevery signal shown changes a plausible next action; decorative percentages were cut.
• Refusals survived the reaction testno moralizing, no apology spiral, an honest adjacent offer where one exists.