Lab 07 · Vendor Claims Tribunal

Farah Qureshi is a senior BA at Sierra Energy Retail, a fictional electricity retailer serving 1.4 million customers, running vendor selection for a customer-communications platform (outage notices, billing reminders, service messages across SMS, email, and WhatsApp). Three vendors responded to the RFP. Procurement wants scores; Farah wants something stronger — an evaluation where every score traces to a claim, and every claim carries a tag saying whether anyone actually verified it. Vendors do not lie, exactly. They round up.

You will need

• A chat AIfree ChatGPT, Claude, or Gemini.
• A spreadsheet for the criteria matrix and claims table.
• Timeabout 60 minutes.
• No real datathe vendors and responses are fictional.

The RFP response excerpts

Condensed to the load-bearing pages. Citations refer to the bracketed page markers.

VENDOR A — "Northlight CX" (excerpts)
[A-p3]  "Northlight supports SMS, email, and WhatsApp Business
        messaging across all plans."
[A-p3]  Footnote 4: "WhatsApp channel generally available in our
        Q3 roadmap; early-adopter programme open now."
[A-p7]  "Fully GDPR compliant; data encrypted at rest and in transit.
        Data residency: EU (Frankfurt) with SOC 2 Type II report
        available under NDA."
[A-p9]  "Native integration with SAP billing systems — certified
        connector, 40+ joint customers."
[A-p12] "Implementation: 10-14 weeks with our accredited partners."

VENDOR B — "Commify" (excerpts)
[B-p2]  "99.99% platform uptime."
[B-p2]  Service terms, p.44 (attached): "Service credits apply below
        99.5% monthly availability."
[B-p5]  "Supported CRM integrations include Salesforce, Dynamics,
        and all major billing platforms via our universal adapter."
[B-p6]  Technical annex: "Universal adapter: nightly CSV export/import
        via SFTP. REST API on the enterprise tier roadmap."
[B-p8]  "Delivery analytics dashboard with per-campaign engagement
        metrics, live since 2023, 200+ customers referenced."

VENDOR C — "Voxcart" (excerpts)
[C-p2]  "AI Reply Assist drafts responses to inbound customer
        messages, cutting agent handle time by up to 60%."
[C-p11] Appendix F: "AI Reply Assist is available as a private
        preview to design partners; production SLAs do not apply."
[C-p4]  "Compliant with data-protection law in our operating regions;
        certified data centres in the EU and US."
[C-p4]  Note 2: "India region: in assessment; customer data for
        Indian deployments is currently hosted in Singapore."
[C-p6]  "Proven at utility scale: 3 of the top 10 EU energy
        retailers send 40M+ messages monthly on Voxcart."

Context you hold from the RFP itself: Sierra is an Indian utility subject to DPDP; its billing system is SAP; outage notices are safety-relevant, so uptime and WhatsApp reach matter.

Steps

1. 1.

   Build the weighted criteria matrix before rereading a single vendor page. Criteria from Sierra's needs, not vendor brochures — for example: channel coverage incl. WhatsApp (weight high — outage reach), billing-system integration, uptime with contractual backing, data-protection fit for India/DPDP, AI capability maturity, implementation risk, evidence of utility scale. Assign weights summing to 100 and write one sentence per criterion on why it earned its weight. Doing this first is the anti-anchoring move — the matrix must not be reverse-engineered from whoever reads best.

2. 2.

   Extract every material claim with AI. Prompt: "From these three RFP excerpts, list every material claim relevant to these criteria: [paste your matrix]. For each claim: vendor, exact quote, page marker, and which criterion it affects. Do not assess truth yet." Check its list against the excerpts — completeness here determines everything downstream.

3. 3.

   Build the claims table. Columns: Claim | Vendor | Page | Criterion | Tag | Evidence needed / found | Score impact. The three tags: verified (independent or contractual evidence exists in the response), vendor-claim (asserted, no backing provided — the default), gap (the response elsewhere contradicts or scopes-down the claim). Nothing scores as fact unless tagged verified.

4. 4.

   Cross-examine each response against itself. This is where the tribunal earns its name: for every impressive claim, hunt the same vendor's footnotes, annexes, and appendices for the scope-down. Prompt per vendor: "Compare this vendor's headline claims against their own footnotes and annexes. List every case where a footnote, annex, or note weakens, scopes, or contradicts a headline claim. Quote both." Verify every hit yourself — page against page. Expect roughly five serious ones across the three vendors.

5. 5.

   Test claims against Sierra's context. A claim can be true and still a gap for Sierra: check every compliance and residency statement against "Indian utility, DPDP applies," and every integration statement against "SAP billing." Add context-mismatch rows to the claims table with tag gap.

6. 6.

   Score the matrix from the claims table only. For each criterion × vendor: score using verified claims at full value, vendor-claims discounted (state your discount rule, e.g., score as if mid-tier until proven), gaps scoring against. Write the score's source row IDs in the cell. A score with no claim rows behind it is an impression, not an evaluation — delete it and re-derive.

7. 7.

   Draft the clarification letter. For every vendor-claim that materially affects ranking, write the question that would convert it to verified or gap: "Please confirm WhatsApp GA date and contractual remedy if the Q3 roadmap slips" — that class of question. This letter is the practical output procurement actually sends.

8. 8.

   Write the recommendation memo. Half a page: ranked vendors with weighted scores, the two or three claims that most moved the ranking, and the sentence every evaluation should contain — which result would change if a specific vendor-claim fails verification.

Verify your work

A complete lab has all of the following:

• •
  A weighted criteria matrix built before claim review, with a rationale line per weight.
• •
  A claims table where every material claim carries a tag and a page citation.
• •
  All five planted overclaims tagged gap with both the headline quote and the contradicting fine print cited.
• •
  Matrix scores that cite claims-table rows, with a stated discount rule for unverified claims.
• •
  A clarification letter and a recommendation memo naming the verification-sensitive claims.