BXUP Track progress & test yourself — free
UI/UX & Agent-Experience Design · Module · ~9 min read
UI/UX & AGENT-EXPERIENCE DESIGN · MODULE

AI-Augmented Research & Synthesis

AI lets one researcher run three times the studies — and one hallucinated quote can undo a year of stakeholder trust. This module teaches research with AI that survives scrutiny: citation-disciplined synthesis, audited AI moderators, precision-checked feedback mining, and synthetic users kept firmly on the hypothesis side of the line.

Ananya is a UX researcher, eight years in, at a Gurgaon GCC with a global research mandate: three times the studies, same headcount. AI is how she keeps up. For a checkout-redesign study she feeds twelve interview transcripts to an assistant, gets back six beautifully organized themes with supporting quotes, drops them into a readout deck, and presents to the product VP on Tuesday.

Theme four says merchants distrust the new payout timeline, supported by a crisp quote: "I check the payout screen every morning because I don't believe the estimate." The VP loves it — it confirms a hunch — and asks to see the clip for the town hall.

There is no clip. Ananya greps every transcript. Nobody said it. The model composited a plausible sentence from the general vibe of three different participants, attributed it to P7, and formatted it exactly like the forty real quotes around it. The theme itself might even be true — but now every claim in the deck is suspect, the VP wants the whole readout re-verified, and the next time Ananya presents, someone will ask, "did a human check this?"

That question — did a human check this? — is the one your research practice must be able to answer with a paper trail. This module is that paper trail, built into every stage of the research workflow.

Warning

Hallucinated quotes are the signature failure mode of AI synthesis. They are fluent, correctly formatted, plausibly attributed, and directionally aligned with the real data — which is exactly what makes them undetectable by skim. The only defense is structural: every quote carries a source citation, and citations get spot-verified against source before anything leaves your desk.

Citation discipline: the non-negotiable core

The fix for Ananya's Tuesday is not "use AI less." It is a synthesis contract you impose on the model and then enforce yourself:

  1. Demand quote-level citations. Every theme must carry verbatim quotes with participant ID and timestamp — "P3, 14:22" — not paraphrases. Instruct the model that a theme without citable quotes must be labeled as inference.
  2. Spot-verify every theme against source. Open the transcript, find the timestamp, read the quote in context. A twelve-transcript synthesis takes 20–30 minutes to verify. That is the price of publishing; pay it every time.
  3. Check context, not just existence. A real quote can still be miscited — sarcasm read straight, a hypothetical read as a report, a moderator's words attributed to a participant. Verification means the quote supports the theme in context, not merely that the string exists.
  4. Mark the verified state. Your readout should be able to say: every quote in this deck was traced to source by a human. Say it on the first slide. It changes how the room treats everything after.
READOUT APPENDIX

checkout study → verification log

41/43QUOTES TRACED TO SOURCE — 2 removed6 themes · 12 transcripts · verified 2026-03-14
  • Theme 1–3, 5–6: all quotes verified in context, timestamps corrected in two places
  • Theme 4, quote "I check the payout screen every morning…" — no source found in any transcript. Removed; theme downgraded to hypothesis pending targeted follow-up
  • Theme 2, P9 quote is real but responds to a leading probe — flagged, retained with caveat

The log is one screen. It is the difference between "AI-assisted" and "AI-laundered" research.

Lab 1 makes this muscle memory: five transcripts, an AI synthesis with two hallucinated quotes deliberately planted, and your job is to catch both without flagging false positives.

Upstream of the data: plans and guides

Synthesis is where AI failures are most famous, but AI enters your research one step earlier — and so does bias. Drafting research questions, screeners, and discussion guides from a study brief is genuinely strong AI territory: models know method trade-offs and produce competent first drafts in minutes.

The catch is that AI-drafted guides skew agreeable. They lead. "How much easier is the new flow?" arrives politely disguised as neutral phrasing, and if you field it, every downstream insight inherits the tilt. So the workflow is symmetric: AI drafts, then AI attacks its own draft — you run a red-team pass with a different prompt (or a different model) whose only job is to find leading questions, double-barreled items, jargon a participant won't share, and screener criteria that quietly select for fans of the product. You adjudicate the findings. Nothing reaches a participant un-red-teamed.

The scaled workflows — and their audits

Three workflows give you the 3x capacity Ananya needs. Each one pairs a scaling mechanism with a named audit, because capacity without audit is how research programs quietly rot.

WorkflowWhat AI doesYour audit gate
AI-moderated unmoderated studiesRuns interview/usability sessions with dynamic follow-ups, any timezone, any hourAudit a sample of sessions for moderator drift before trusting any of it
Session analysis at scaleAuto-transcribes, extracts usability issues with severity and video-clip linksVerify every issue against its clip before it reaches the fix list
Feedback miningClassifies thousands of reviews/tickets/NPS verbatims into themes with countsSample-audit ~20 items per theme for precision before roadmap influence

AI-moderated studies (Maze-class moderators, Outset, UserTesting AI) solve a real problem — reaching participants across timezones without 2 a.m. calls — and industry data says roughly 69% of researchers now use AI somewhere in their projects. But the moderator is a model: it can ask leading follow-ups, miss a signal a human would chase, or wander off-guide entirely. You design the guide, set explicit follow-up boundaries (what it may probe, what it must not), and then audit a sample of session recordings for drift. Signals worth hunting in an audit:

  • Leading follow-ups"so that was frustrating?" instead of "how did that feel?"
  • Missed goldminesa participant volunteers a workaround and the moderator moves to the next scripted question.
  • Off-guide tangentsfriendly chit-chat that burns session minutes without producing data.
  • Premature closureaccepting a vague answer ("it was fine") where the guide demanded a concrete example.

Two India-specific realities belong in your study plans. Global research panels are thin on Indian consumer segments, so bring-your-own recruitment remains the workable path for most studies here — AI moderation helps precisely because it removes the timezone tax on a self-recruited panel. And multilingual sessions — Hindi, Hinglish, regional languages — are an emerging advantage of AI moderation, but transcription and moderation quality in Indic languages varies sharply by tool. Run a pilot session in the target language and audit the transcript word-by-word before committing a study to it; a moderator that mis-transcribes code-switched Hinglish will corrupt every downstream synthesis silently.

Session analysis at scale turns twenty usability recordings into an issues list with severity and clip links in an afternoon. The clip link is the whole game: an issue you can't watch is an issue you can't trust. Verify clip-by-clip; discard or re-scope what the video doesn't support.

Feedback mining is the highest-volume workflow: a structured-output schema (theme, sentiment, severity, representative quote, source ID) applied over thousands of verbatims. Before any theme count influences a roadmap, sample about twenty classified items per theme and score precision — is this item really this theme? A theme at 70% precision with 400 items is a very different roadmap input than one at 98%, and only the audit tells you which you have. Lab 3 runs this end to end on a thousand app-store reviews.

Pro Tip

Write the audit into the study plan before you run the study — "sample size for moderator audit: 3 of 15 sessions; precision audit: 20 items per theme, threshold 90%." Deciding audit depth after seeing convenient results is how motivated reasoning gets institutionalized.

Synthetic users: hypothesis generators, never evidence

Somewhere this quarter, someone will suggest skipping recruitment: "just ask ChatGPT what our users think." The suggestion isn't crazy — persona-conditioned models produce eerily fluent user-like responses — which is exactly why it needs a hard boundary rather than a sneer.

NN/g studied synthetic users directly and documented the failure that matters: persona-conditioned models are systematically over-optimistic. In their documented cases, synthetic users responded enthusiastically to concepts that real users later rejected — a concept-test false positive that would have greenlit a dud. Models also lack the context that produces real insight: the workaround born of a specific org policy, the distrust earned by last year's outage, the constraint nobody mentions because it's too obvious to say.

LEGITIMATE

As hypothesis generator

Sharpen research questions before fielding. Stress-test a discussion guide by roleplaying a difficult participant. Generate proto-personas to argue with. Explore the space of possible objections so your real study probes them. Every output is labeled hypothesis and scheduled for testing against reality.

ILLEGITIMATE

As evidence

Concept-test "results" from synthetic panels. Persona quotes in a readout. "Users told us" where no user was involved. Any synthetic output that reaches a decision-maker without the word synthetic attached. This is fiction with a citation format — Ananya's hallucinated quote, industrialized.

Lab 4 makes the boundary visceral: you run the same concept test against synthetic users and against provided real-user data, document the divergence — including the seeded optimism failure — and write your team's one-page synthetic-user policy.

Artifacts that stay honest: personas, journeys, and the repository

Two more practices close the loop between studies.

Evidence-linked personas and journey maps. Personas drift into fiction the moment they detach from data — AI just accelerates the drift by making fictional detail cheap. The rule from Module 1 applies with full force: every claim on a persona or journey map carries an evidence link to real synthesized research (your citation-disciplined themes, your audited feedback mining). AI is excellent at the refresh: feed it the latest verified synthesis and the current persona, ask for a diff — which claims are newly supported, which are contradicted, which are stale. Run that stale-claim sweep on a cadence, quarterly at minimum, and date-stamp every claim.

The research-repository RAG. Once your verified studies accumulate, "what do we know about onboarding drop-off?" should be answerable in seconds — through a repository assistant: Dovetail-class magic search, a Notion AI Q&A, or an API-built RAG over your corpus. Four design rules make it trustworthy rather than merely convenient:

  • Citations on every answereach claim links to the study and quote it came from, so any stakeholder can follow the chain to source.
  • Recency weightinga 2026 study outweighs a 2022 finding on the same question, and the answer says which era it's drawing from.
  • Refusal on thin evidencewhen the corpus holds one weak signal, the honest answer is "we don't know enough," not an improvised synthesis dressed as institutional knowledge.
  • Curated ingestionverified syntheses enter the corpus; raw unaudited AI output never does. A repository that ingests unverified synthesis becomes a hallucination-laundering machine: the fake quote gets retrieved, re-cited, and hardens into fact.

Try it now

  1. Verify one old synthesis. Take any AI-assisted summary you've produced or received in the last month and trace three of its claims back to source material. Log what you find — most people catch at least one paraphrase drift on their first attempt.
  2. Red-team a guide. Feed a past discussion guide to an assistant with one instruction: find every leading question, double-barreled item, and biased framing. Rewrite the two worst offenders neutrally.
  3. Draft your synthesis contract. Write the five-line instruction block you will paste into every future synthesis prompt: quote-level citations with participant ID and timestamp, inference labeled as inference, no composite quotes, unknowns stated as unknowns. Save it where you'll actually reuse it.
Key Takeaway:

AI gives research capacity; only verification gives it authority. Citations traced to source, moderators audited for drift, theme counts precision-checked, synthetic output labeled hypothesis — that paper trail is what lets you answer "did a human check this?" with yes, every time, and mean it.

This is the free reading view. Open BXUP to track your progress, run the hands-on labs, and prove it with original, exam-pattern assessments.
© Anvi Innovate · BXUP — Be Exponentially Upskilled · BXUP is an independent learning platform, not affiliated with or endorsed by AWS, Microsoft, Google, NVIDIA, IBM, Salesforce, Databricks, Oracle or Anthropic. All trademarks belong to their respective owners. · bxup.in