Pre-Screen Designs With an AI Heuristic Review

Workflow area: Ideation & Critique · Tier: Core · Hands-on time: ~30 minutes

Heuristic evaluation was always the cheap, fast, imperfect method: expert judgment against known principles, catching perhaps a third to a half of real problems while flagging some that users never hit. A vision model running the same heuristics inherits all of that — including the false-positive rate — at a fraction of the cost. That makes the economics excellent and the epistemics dangerous. Used before research, it clears the underbrush so studies find the deep problems. Used instead of research, it launders "an AI guessed" into "an evaluation found." This playbook keeps it firmly on the right side of that line.

When to use — and when not to

• Use it before usability testingfixing the obvious violations first means sessions surface the problems only real users can reveal, instead of re-discovering a missing back button.
• Use it on flows without research budgetinternal tools and admin screens that will never get a study still deserve a structured pass.
• Use it to triage a large surfacetwenty screens through the model tells you which three deserve your expert attention first.
• Skip it as evidence in a stakeholder fight"the AI found 14 issues" persuades no one who matters and deserves not to; bring clip evidence from pb-15 instead.
• Skip it for novel interaction patternsheuristics encode convention; a genuinely new pattern will be flagged as a violation precisely because it is new, and the model cannot tell innovation from error.

The recipe

1. 1.

   Capture the flow in order, at real fidelity. Export screens in sequence — entry, core task, confirmation — at readable resolution. Include the states you have (empty, error) because state gaps are half of what heuristic review catches. Number the files; the model will cite them.

2. 2.

   Run the review with heuristics named and scope pinned. In Claude or ChatGPT with images attached:

   Copy

   You are performing a heuristic evaluation of this flow.
   Context: [PRODUCT, USER, THE TASK THIS FLOW SERVES]
   Screens attached in task order, numbered.
   
   Evaluate against:
   - Nielsen's 10 usability heuristics (name the specific heuristic
     per finding)
   - [iOS HIG / Material Design / your platform] conventions for
     navigation, controls and feedback
   
   For each finding:
   - screen number and the specific element (location on screen)
   - heuristic or guideline violated, by name
   - what the user problem would be, as a concrete scenario
   - severity 1-4 (Nielsen scale: cosmetic → catastrophic) with
     one line of reasoning
   Rules: findings must reference visible evidence in the screens.
   Do not assume behavior you cannot see — flag "cannot assess
   [hover states / motion / loading] from static images" explicitly.
   No praise, no summary of what works.
   

3. 3.

   Interrogate the suspicious findings. For any finding that smells off, push back: "Screen 3, finding 7 — the model says the filter state is invisible. Describe exactly what pixels indicate filter state." The model either points at real evidence or retracts. This one habit removes most false positives before they cost anyone time.

4. 4.

   Validate severity yourself — every single rating. The model rates severity by rule; you rate it by knowing your users, their frequency on this flow, and the cost of the error. Re-grade the list:

   Copy

   #  SCREEN  FINDING                      AI SEV  MY SEV  NOTE
   3  2       No feedback after "Save"        3       4    Core action, hit
                                                           dozens of times/day
   7  3       Filter state not visible        3       1    Power-user screen;
                                                           state shown in URL
   9  4       Destructive delete lacks        4       4    Confirmed; ship
              confirmation                                 blocker
   

   Findings the model could not assess from statics (motion, latency, focus order) go to a separate list for pb-31's artifact-side pass or real testing — not silently dropped.

5. 5.

   Ship the pre-screen as prep, labeled as such. The output is a fix list for the violations you confirmed, plus a research agenda: the ambiguous findings become tasks and questions for your next usability study. Title the document "AI-assisted heuristic pre-screen" — the label is the honesty mechanism that keeps it from being quoted as user research in three months.

Make it yours

• Attention Insight for the perception layer: run predicted attention heatmaps on key screens alongside the heuristic pass — a critical CTA in a predicted attention dead zone corroborates a visibility finding from a different method.
• Two-model cross-examination: run the same screens through Claude and ChatGPT; findings both models raise independently are your highest-confidence violations, and single-model findings get extra skepticism.
• House-rules edition: append your design-system rules and past research findings to the heuristics list — "we know from March research that users miss toasts" turns a generic pass into one that remembers your product's history.

Quality check

• Every finding names its heuristic and its pixelsa violation that cannot cite a principle and a screen location is a vibe.
• You re-graded every severitythe AI's severity column untouched means the judgment step did not happen.
• False positives were discarded and countedthe discard rate calibrates how much to trust the next run.
• "Cannot assess" items went somewhere realmotion, focus and latency findings routed to testing or the a11y gate, not dropped.
• The document says pre-screen, not evaluationand nothing in it is phrased as what users do; the model has never met your users.