Run a real-web research sprint: compare three actual meeting-notetaker tools against a synthetic company's requirements using AI deep research plus manual source checks, and produce a one-page recommendation matrix with citations — where at least two AI-found claims are traced to primary sources before they enter the matrix.
Nikhil Rao is the operations manager at Cobalt Advisory LLP, a fictional 60-person consulting firm. Partners want an AI meeting notetaker firm-wide; Nikhil owns the recommendation. This lab is different from the others in one important way: the research targets are real products on the real web, researched live — because vendor comparison is a task where AI genuinely accelerates you and genuinely burns you. Models cite stale pricing, confuse plan tiers, and repeat vendor marketing as fact. The discipline you are building: AI finds the claims fast, and you promote a claim into the decision matrix only after tracing it to a primary source — the vendor's own current page or documentation. The requirements sheet below is synthetic, and like most requirement sheets written by committees, it does not entirely agree with itself.
From Cobalt's ops committee. Paste-ready.
COBALT ADVISORY - MEETING NOTETAKER REQUIREMENTS (v2, ops committee)
CONTEXT: 60 staff, ~45 will use the tool. Google Workspace shop;
meetings on Google Meet, some client calls on Zoom and Teams.
Client work is confidential; several clients are banks with strict
vendor-security expectations.
MUST HAVE:
M1. Works with Google Meet AND Zoom (Teams nice-to-have).
M2. Transcription plus AI summaries and action-item extraction.
M3. Admin controls: central workspace management, ability to
disable recording for designated client engagements.
M4. Data handling: vendor must state where data is stored, offer
data deletion, and not train public AI models on our content
by default. SOC 2 or equivalent attestation required.
M5. Participants must be clearly notified when the notetaker
is active. Compliance requires that external client calls
are recorded by default so nothing is missed.
M6. Budget: total spend must not exceed Rs 8,000 per user per
year. At 45 users this is a total budget of Rs 4,80,000
per year.
NICE TO HAVE:
N1. Hindi + English transcription support.
N2. CRM/task-tool integrations.
N3. Ability to query past meetings ("what did we promise client X").
POLICY NOTE: Firm policy 7.3 states no client call may be recorded
without written client consent obtained in advance per engagement.
Audit the requirements before researching anything. Read the sheet the way you audited data in earlier labs: check the arithmetic, and check the requirements against each other and against the quoted policy. Committee documents accumulate contradictions the same way spreadsheets accumulate typos. Write down anything that cannot be satisfied as written — you will need a resolution from the "committee" (in this lab, you make the sensible call and document it) before a fair comparison is possible.
Pick the three candidates. Ask your AI: "List the leading AI meeting notetakers that support both Google Meet and Zoom, suitable for a 45-60 person professional-services firm. For each: one line on positioning." From its list, pick three well-known tools (for example Otter, Fireflies, and Fathom — but any three real, currently available candidates are fine). Record why you picked each — the shortlist rationale is part of the deliverable.
Run the broad research pass. For each candidate, ask: "For [tool], as of today: business/team plan pricing per user, Google Meet and Zoom support, admin controls, data storage location and retention, whether customer content is used for AI training by default, security attestations (SOC 2 etc.), language support including Hindi, and consent/notification behavior when joining calls. Give a source link for every claim." Save all output to your working doc — this is raw material, not truth yet. Deep-research modes produce this in one shot; standard search modes may need one prompt per tool.
Build the matrix skeleton. In your spreadsheet: rows = requirements M1–M6 and N1–N3 (as resolved in step 1); columns = the three tools; every cell gets a value and a status tag — VERIFIED (you saw a primary source), AI-CLAIMED (from step 3, unverified), or UNKNOWN. Fill it entirely from step-3 output first, so every cell starts as AI-CLAIMED or UNKNOWN. Watching the tags flip to VERIFIED as you work is the honest picture of how much you actually know.
Trace the claims that decide the outcome — minimum four. You will not verify every cell in an hour; verify the ones the decision turns on: pricing for the plan tier you would buy (both finalists), the AI-training-on-your-data default (all three — this is M4's sharpest edge and the single most commonly misreported fact in this category), and one admin/consent control claim. For each: open the vendor's own pricing or security/trust page, confirm or correct the AI's claim, and record the URL and today's date in the cell note. Expect at least one AI claim to be outdated or plan-tier-confused — finding it is a feature of the lab, not a detour.
Convert prices and check the budget line. Vendor pricing is in dollars; the budget is in rupees. Convert at the current rate (note the rate and date), compute per-user-per-year for the realistic plan tier, and test against the corrected budget figure from your step-1 audit. Per-user SaaS pricing times 45 users compounds small errors fast — this is a two-minute spreadsheet check that AI-generated comparisons routinely fumble.
Score, decide, and write the one-pager. Score each tool: musts as pass/fail (a failed must is disqualifying, and a must that no tool passes goes back to the committee as a finding), nice-to-haves as 0–2. Then write the one-page recommendation: the pick, the two-line rationale, the matrix with status tags visible, the requirements issues you found in step 1 and how you resolved them, and a "verify before contract" list — every decision-relevant cell still tagged AI-CLAIMED, plus a pilot plan (5 users, 2 weeks) as the risk control.
Date-stamp and shelf-life the research. Add the final line: "Pricing and capability claims verified [date]; re-verify before signature — SaaS pricing and AI-training policies change without notice." Vendor research decays in weeks. A dated matrix with an explicit shelf life is professional; an undated one is a future incident.
The vendor's own current pricing page beats any AI summary, any comparison blog, and any review site — when they disagree, the vendor page as of today is the only citable source.
AI research gives you claims at speed; primary sources give you facts you can sign. Tag every cell with its verification status, trace the claims the decision turns on, audit the requirements before comparing anything against them — and date-stamp the result, because vendor truth has a shelf life.