BXUP Track progress & test yourself — free
UI/UX & Agent-Experience Design · Playbook · Advanced · ~5 min read
UI/UX & AGENT-EXPERIENCE DESIGN · PLAYBOOK · ADVANCED

Prototype AI Features with Real Model Responses

Wire your prototype to a live LLM endpoint so usability tests exercise variable outputs, real latency and genuine failure states. Static happy-path mocks systematically mislead AI-feature testing — participants react to your best-case screenplay, not to the actual distribution of things the model will do to them.

Workflow area: Figma AI & Prototyping · Tier: Advanced · Hands-on time: ~45 minutes

A conventional prototype fakes its backend and loses nothing: the confirmation screen always confirms. Fake an AI feature's backend and you lose the entire phenomenon under test. The mock shows one perfect response at zero latency; the shipped feature streams variable-quality answers after an uncomfortable pause, sometimes hedging, sometimes wrong, occasionally refusing. Participants who "loved the assistant" in a mocked test were rating your copywriting. Figma Make closed the gap: a prototype can now call a real model with your real system prompt, which means your test sessions finally sample the same distribution your users will live in — including the sessions where the model embarrasses itself, which are the sessions you learn from.

When to use — and when not to

  • Use it before committing to an AI feature's interaction modelchat vs. inline vs. structured input (pb-42) feels totally different against real latency and variance; decide with live evidence.
  • Use it to test trust surfacescitation anatomy (pb-36), uncertainty displays, failure states (pb-48) only produce valid findings when the responses are genuinely variable.
  • Use it for stakeholder truth-tellinga live demo including an imperfect response sets honest expectations that a sizzle-reel mock actively sabotages.
  • Skip it when live wiring is impracticalno API budget, locked-down data, or a model that doesn't exist yet: run a Wizard-of-Oz protocol (a human plays the model from a response playbook) rather than regressing to happy-path mocks.
  • Skip it for pure layout explorationif the question is visual hierarchy, not AI behavior, static frames are cheaper and fine.

The recipe

  1. Define the behavior you're testing before wiring anything. Write down the two or three questions the prototype must answer: Do users notice the citation chips? What do they do during a 6-second generation? Does the refusal state read as safety or brokenness? These questions drive the system prompt, the seeded scenarios and the test protocol — wiring without them produces an impressive demo and no findings.

  2. Build the shell in Figma Make from your designed frames. Prompt Make with your static frames as the source and the interaction contract stated explicitly:

    Turn this design into a working prototype of [FEATURE]. The
    AI-response area must be wired to a real model call, not canned
    text. Requirements:
    1. Input: [chat box / inline action / madlib — per the design].
    2. While waiting: show the designed loading state (streaming
       indicator after 400ms, skeleton before).
    3. Render model responses with: [citation chips / hedge banner /
       feedback buttons — per the design].
    4. Include a visible "Stop generating" affordance.
    5. On API error or timeout (>15s): show the designed failure
       state, not a blank or a spinner forever.
    Use the design system components in the file. Production code is
    out of scope — this is a test instrument.
    
  3. Install the feature's real system prompt — this is the fidelity core. The model behind the prototype must behave like the feature, not like a helpful generalist. Write the system prompt with the feature's role, tone (from your pb-27 voice system), capability boundaries and refusal rules, and iterate on it until responses feel like your product. If the production feature will use retrieval, approximate it: paste representative source content into the prompt context so answers cite the kind of material the real feature would.

  4. Seed the failure states deliberately. Left alone, a strong model succeeds too often to test your failure design. Force the distribution: add prompt rules that trigger refusals on defined topics, prepare test tasks that hit the feature's genuine weak spots, and if needed script one deterministic failure (a timeout you inject, a scenario the context can't answer) so every participant meets at least one non-happy path. A prototype that cannot fail cannot test failure UX — build the failure in.

  5. Pilot against variance before real participants. Run your own tasks through five times. You're checking: does response variance stay inside what the UI handles (length overflow, weird formatting, markdown the renderer chokes on)? Do latency and streaming behave? Does the seeded failure fire reliably? Fix the shell's tolerances now — a rendering bug discovered mid-session costs you the participant.

  6. Test with a protocol built for non-determinism. Every session will differ, so define task success over outcome classes, not exact outputs: "participant reaches an answer they can verify" rather than "participant sees response X." Log which response class each participant actually got (good / hedged / partial / failed) next to their behavior, and brief the moderator for live-model surprises: when the model fails in an unplanned way, follow the participant's reaction — that moment is data, usually the best in the session. Full protocol treatment is pb-50; this playbook gets the instrument right.

Make it yours

  • Claude API for controlled variance: wire the prototype to the Claude API directly and use temperature plus system-prompt strictness to tune how variable responses are — tight for testing one designed state, loose for sampling the real distribution.
  • Response-log replay: capture every live response during piloting into a log, then add a replay mode that serves logged responses with realistic streaming — deterministic sessions when you need cross-participant comparability, live mode when you need truth.
  • Wizard-of-Oz fallback kit: when wiring is impossible, prepare a human-operator playbook — response templates per scenario including hedges and refusals, typed live with realistic delay — same test questions, lower fidelity, still far above a static mock.

Quality check

  • The system prompt makes the model behave like the featurecapability limits, tone and refusal rules match your spec; a generic assistant behind your UI tests someone else's product.
  • Latency and streaming are real or realistically simulatedperceived-performance findings from an instant prototype are void.
  • At least one failure path fires per sessionby seeding or scripting; if every participant sailed through, you tested the happy path with extra steps.
  • The shell survives variancelong responses, odd formatting and refusals render inside the designed states; piloting proved it.
  • Findings separate model behavior from UX behavioryour notes distinguish "the model answered badly" from "the UI handled it badly"; only the second is a design finding, though the first informs pb-48.
Warning

Live-wired prototypes send participant input to a real API — tell participants an AI model processes their input, keep API keys out of shared prototype links, scrub any client or personal data from prompt context, and use your organization's approved API tenant for anything non-public.

Key Takeaway:

Test the distribution, not the demo. A prototype wired to a real model — with the feature's true system prompt and at least one failure you made sure would happen — is the only instrument that tells you how your AI feature's UX performs when it isn't being flattered.

This is the free reading view. Open BXUP to track your progress, run the hands-on labs, and prove it with original, exam-pattern assessments.
© Anvi Innovate · BXUP — Be Exponentially Upskilled · BXUP is an independent learning platform, not affiliated with or endorsed by AWS, Microsoft, Google, NVIDIA, IBM, Salesforce, Databricks, Oracle or Anthropic. All trademarks belong to their respective owners. · bxup.in