BXUP Track progress & test yourself — free
Product Management · Module · ~10 min read
PRODUCT MANAGEMENT · MODULE

Discovery & Research with Verification Discipline

AI turns forty hours of synthesis into four — and will invent a customer quote with the same fluency it summarizes a real one. This module builds the discovery workflows where every theme carries a checkable citation, every competitive claim carries a source and a date, and hallucinated evidence dies before it reaches a decision.

The scenario — the quote that never existed

Rohan is a PM at a Bengaluru GCC, six years in, recently handed global ownership of the onboarding surface for a US-headquartered HR platform. Ownership came with a synthesis problem: fourteen customer interviews across three time zones, six hundred NPS verbatims, and a Monday deadline for a discovery readout to the VP in Austin.

He did what the tooling invites: uploaded all fourteen transcripts to an assistant and asked for themes with supporting quotes. The output was superb — five crisp themes, each with three verbatim quotes, customer names attached. Theme three was the headline: "Admins don't trust the bulk-import preview," supported by a devastating quote from a named enterprise customer: "I export to Excel first because I've been burned by the preview twice."

The VP loved the readout. She repeated that quote — customer name and all — on a call with that customer's account team. The account team checked the transcript to get context. The quote wasn't there. Nothing like it was there. That customer had barely discussed imports. The model had synthesized a plausible quote from the general shape of the feedback and attributed it to a specific person, and Rohan had shipped it upward without opening a single transcript.

The theme, it later turned out, was real — other customers had said similar things. That's what makes this failure mode dangerous: the model is usually directionally right, so spot-checks feel unnecessary right up until a fabricated quote with a real name on it reaches the customer it was fabricated about.

Warning

Hallucinated quotes are the signature failure mode of AI research synthesis. They are formatted identically to real quotes, attributed confidently, and usually plausible — which is precisely why unverified synthesis is worse than no synthesis: it manufactures false evidence with real names attached.

What You'll Learn

  • Interview synthesis with citation disciplinethemes that carry quote-level citations you spot-verify against source
  • Meeting capture to research notesthe notetaker-to-repository pipeline with a PM review pass
  • Feedback mining at scalethousands of verbatims into counted themes, kept honest by sample audits
  • Call-library miningsales and support recordings as a product-signal source with clip-level evidence
  • Competitive research with a verification ledgerdeep research where every claim has a source URL and a freshness date
  • Synthetic users, research RAG, and usability synthesisthe boundaries that keep each honest

Interview synthesis — the citation contract

The fix for Rohan's failure is not "don't use AI for synthesis" — synthesis is one of the highest-leverage PM uses of AI, and at GCC scale it's the difference between drowning and steering. The fix is a citation contract imposed on the model and enforced by you:

  1. Ingest with identifiers. Load transcripts into a project with stable labels (P1–P14, with a key mapping to real names kept outside the tool if confidentiality requires).
  2. Demand quote-level citations. The synthesis prompt requires every theme to cite participant ID and a verbatim quote — and explicitly instructs: quotes must be copied exactly from the transcript; if no verbatim quote supports a theme, say so rather than paraphrasing.
  3. Spot-verify before anything ships. For every theme, open one cited source and find the quote with search. Verbatim instruction or not, models paraphrase and occasionally invent — the instruction reduces the rate; only the check catches the remainder.
  4. Audit the orphans. Ask the model what evidence contradicts each theme, and which participants said nothing about it. This surfaces both planted-contradiction problems (two customers who disagree) and over-generalized themes built from one loud voice.
  5. Log the audit. A small table — theme, cited source, verified yes/no, notes — attached to the synthesis doc. Lab 1 grades exactly this artifact.

The check costs ten minutes on a fourteen-transcript synthesis. Rohan's incident cost a customer-facing apology and a quarter of credibility with the Austin VP. That is the trade.

Dedicated research tools change the economics but not the contract. Dovetail-class research platforms and NotebookLM ground their answers in your uploaded sources and link back to them — NotebookLM in particular refuses to answer beyond the corpus, which makes it a strong default for "what did our research actually say?" queries. But grounding reduces invention; it doesn't eliminate misreading, and no tool exempts you from opening the source on load-bearing quotes.

Meeting capture — the pipeline, not the transcript

One interview synthesized well is craft; every call captured systematically is infrastructure. The pipeline has three stages, and the middle one is where PMs skimp:

  • Capturea notetaker on every user and customer call. Fireflies-class free tiers cover a normal PM's call volume; Zoom AI Companion if that's what's sanctioned. Raw transcripts are the artifact; auto-summaries are convenience, not record.
  • Templated extractioneach transcript through a fixed prompt extracting pains, verbatim quotes with timestamps, jobs-to-be-done, feature mentions, and follow-ups, into the same structure every time. Consistency of structure is what makes a hundred calls minable later.
  • The two-minute PM reviewbefore the extraction enters your research repo, you skim it against your memory of the call. You were there; the model wasn't. Wrong emphasis, missed sarcasm, and a mangled competitive mention take two minutes to catch now and are unfindable later.

Feedback mining at scale — counted themes, sampled honesty

Tickets, NPS verbatims, and app-store reviews arrive in the thousands, which is beyond reading and exactly the scale where AI classification shines — and where a subtle failure mode appears: the model's theme labels drift, absorb, and blur, so "confusing setup" quietly swallows "broken setup," and your counts become fiction.

The working shape, which Playbook 17 operationalizes:

  • Fix the schema firstclassify into a structured output: theme (from a starter taxonomy you seed, extensible), severity, segment, representative quote, source ID. Schema-first beats "find themes" because it makes every run comparable.
  • Batch, don't marathonprocess in batches of a few hundred; long single runs degrade and drift mid-context.
  • Count, then sample-auditfor each of your top themes, pull 20 classified items and check them by hand. Precision below roughly 90% means the theme label is doing too much work — split it and rerun. This 20-item audit is the entire difference between "AI says onboarding complaints are up 40%" being a finding versus a liability.
  • Keep quotes traceableevery representative quote carries its source ID, for the same reason as interviews: anything that looks like evidence must be checkable in one click.

The same discipline extends to call-library mining — querying Gong-class recordings or exported transcripts for "what do churned customers say about onboarding?" — with clip-level links as the citation equivalent, and to usability-test synthesis, where AI drafts the issue list with severity rankings and you verify every P0 and P1 against the actual footage before it drives a sprint.

Competitive research — the verification ledger

Deep-research modes in ChatGPT, Claude, Gemini, and Perplexity will produce a competitive brief in twenty minutes that would have taken two days in 2023 — browsing, citing, and structuring as they go. They will also, with total confidence, report a competitor's 2024 pricing as current, describe a feature from a help-center page that no longer exists, and blend a rumor from a forum post into a factual claim. Deep research compresses the reading, not the verifying.

The workflow that survives exec scrutiny turns the brief into a claims table and the claims table into a verification ledger:

COMPETITIVE BRIEF · CLAIMS LEDGER

scheduling-competitor deep-research run

23 claimsEXTRACTED — top 10 manually verified6 verified · 2 stale · 1 unsupported · 1 wrong
  • "Launched shift-swap marketplace Q1 2026" — vendor changelog, checked 04 Aug
  • "SOC 2 Type II since 2024" — trust page, checked 04 Aug
  • "Starts at $4.50/user/mo" — STALE: pricing page now shows $6, tiers restructured
  • "No India data residency" — UNSUPPORTED: no source found either way; downgraded to open question
  • "Deprecated their API v1" — WRONG: v1 sunset announced, not executed; misread of a blog post

Every claim: source URL + freshness date. Top 10 by decision-weight: manually re-verified before the brief circulates.

The rules that make this work:

  • Every claim gets a source URL and a freshness dateno source, no claim; the model must show where each assertion came from and when that source was current.
  • Rank claims by decision-weight, verify the top ten by handyou cannot check everything; you can always check the claims your pricing or roadmap decision actually rests on.
  • Record the failures, not just the survivorsthe ledger rows that say STALE and WRONG are the point. They calibrate your team's trust in AI research, and they are your answer when someone asks "how do you know this brief is solid?"
  • Date the brief itselfcompetitive facts decay; a brief without a "verified as of" date becomes misinformation on a schedule.

Run weekly signal briefings the same way at lower intensity: scheduled digests of competitor releases and pricing changes, links mandatory, curated by you before anything circulates.

Pro Tip

In lab 2 you will run this exact workflow on a real product category, and some of your AI's claims will fail verification — the lab is designed so the ledger has failure rows. Keep that ledger. A documented "here's what the AI got wrong and how I caught it" is the most persuasive verification artifact you can show a skeptical stakeholder or an interviewer.

Two accelerants with hard boundaries

SYNTHETIC USERS

Hypothesis generator, never evidence

Persona-conditioned critique — "as a shift supervisor at a 200-store retailer, react to this concept" — is genuinely useful before research: it sharpens interview questions and surfaces angles you hadn't considered. The boundary: its output is a guess shaped like feedback. No simulated user ever validates demand, ranks a pain, or appears in a readout as evidence. If a synthetic critique feels compelling, that's a research question, not a finding.

RESEARCH-REPOSITORY RAG

Institutional memory with citations

Past reports, interviews, and syntheses embedded and queryable — "what do we know about admin onboarding?" — via NotebookLM, Dovetail's ask features, or Claude Projects over your corpus. The disciplines: answers must cite the underlying document; recency is weighted or at least visible, because 2024 findings about a redesigned flow are anti-knowledge; and thin evidence gets a refusal, not a confident synthesis from two stray mentions.

A useful comparison for choosing where each tool earns a place:

TaskGood defaultWhyWatch for
Multi-transcript synthesisAssistant project + citation contractFlexible, free, forces your disciplineInvented quotes — always spot-verify
"What did our research say?"NotebookLM / research repo RAGSource-grounded, refuses beyond corpusStale findings surfacing without dates
Verbatim mining at scaleAPI batch or Dovetail-class toolSchema + counts at thousands-scaleTheme drift — 20-item sample audits
Competitive claimsDeep research + verification ledgerCompresses reading dramaticallyStale and unsupported claims — verify top 10
Pre-research critiquePersona-prompted assistantCheap hypothesis generationTreating simulated feedback as evidence

Try it now

  1. Run the citation contract. Take your two most recent customer-call transcripts, synthesize them with quote-level citations required, then spot-verify every quote with search-in-page. Count the discrepancies — even paraphrases count.
  2. Audit one theme at scale. Pull 50 recent tickets or NPS verbatims, classify them into themes with a fixed schema, then hand-check 20 items from the largest theme and compute your precision.
  3. Start a ledger. Run one deep-research query on a competitor you know well, extract the ten most decision-relevant claims into a table with source URLs, and manually verify five. Record survivals and failures — your first verification ledger.
Key Takeaway:

AI synthesis is a claims generator, not an evidence source — the evidence lives in your transcripts, tickets, and verified sources. Impose the citation contract, sample-audit the counts, ledger the competitive claims, and let nothing that looks like evidence leave your desk unchecked. The discipline costs minutes; shipped fabrications cost credibility you don't get back.

This is the free reading view. Open BXUP to track your progress, run the hands-on labs, and prove it with original, exam-pattern assessments.
© Anvi Innovate · BXUP — Be Exponentially Upskilled · BXUP is an independent learning platform, not affiliated with or endorsed by AWS, Microsoft, Google, NVIDIA, IBM, Salesforce, Databricks, Oracle or Anthropic. All trademarks belong to their respective owners. · bxup.in