Before the claims document-AI pilot spends a rupee, score the data it depends on: quality, coverage, ownership, legal usability, runtime retrievability. The landscape hides two pilot-killers — a consent basis that misses the intended purpose, and a critical field null a third of the time. Find both before the pilot finds them for you.
The business case from Lab 11 cleared review with one condition, added by the CFO's analyst: "Subject to data readiness." Smart condition. Most dead AI pilots did not die of bad models — they died of data that was legally unusable, mostly missing, or unfetchable at runtime, discovered in week six instead of week zero. Divya's final pre-pilot task at Meridian General Insurance is the readiness assessment: five dimensions, scored against evidence, with a prerequisite register naming who must fix what before the pilot starts.
Three evidence sets: profiling extracts, catalog entries, and consent records. The pilot's need: at claim intake, retrieve the claim's document set and policy context, extract fields, and check completeness against cover type.
PROFILING EXTRACT — run last week, production copies
P1 claims_core.claims (12 mo): 108,411 rows; claim_id 100% populated,
unique; cover_type 99.2% populated; incident_date 98.8% valid
P2 docstore.claim_docs (12 mo): 391,202 rows; file present 99.7%;
doc_type field 34% NULL (unlabelled uploads); OCR-able 91%
P3 policy_admin.policies: policy_id match rate to claims: 99.1%;
cover_details JSON schema v2 (78% of rows) vs legacy v1 (22%)
P4 crm.customers: email 88% populated; mobile 97%; consent_flags
present for 100% of rows created after Aug 2023, 61% before
CATALOG ENTRIES (from the enterprise data catalog)
C1 claims_core.claims — owner: Claims Ops Data Team (named steward:
R. Batra); refresh: real-time; classification: confidential
C2 docstore.claim_docs — owner: BLANK ("legacy migration 2021");
refresh: real-time; classification: confidential-PII
C3 policy_admin.policies — owner: Policy IT (steward: S. Menon);
refresh: nightly batch; classification: confidential
C4 crm.customers — owner: CRM CoE; refresh: hourly;
classification: confidential-PII
CONSENT RECORDS (privacy office summary)
N1 Claims intake consent (all claimants, at FNOL): processing of
claim documents "for claim registration, assessment, and
settlement of the claim"
N2 Marketing consent (opt-in, 41% of customers): communications
and offers
N3 There is NO consent or notice covering use of claim documents
for "automated analysis, model evaluation, or service
improvement"; privacy office confirms none was ever collected
N4 Retention: claim files retained 8 years post-settlement
(regulatory); CRM data retained while relationship active
Write the pilot's data demand before scoring anything. One paragraph plus a table: which datasets, which fields, fetched when (runtime at FNOL vs batch), for what purpose (extraction + completeness checking now; eval-set construction and accuracy measurement too — write this down, it matters in step 4). Scoring readiness without a stated demand is how assessments turn into dataset beauty contests.
Set up the scorecard. Rows: the four datasets. Columns: the five dimensions — Quality (nulls, validity, duplicates), Coverage (does it span the cases the pilot will see), Ownership (named, reachable steward), Legal usability (basis covers the pilot's purposes), Runtime retrievability (available at the moment the process needs it, at the freshness it needs). Score each cell RED / AMBER / GREEN with a one-line evidence citation (P/C/N line). No cell scores without a citation.
Score quality and coverage from the profiling extract. Do this pass yourself, then ask the AI to re-derive it: "Score data quality and coverage for this AI use case from these profiling lines. Flag any field whose null rate or schema split would materially damage extraction or completeness-checking." Reconcile. The extract contains one field that should stop you cold — and one schema split (P3) that is a real but manageable amber. Know which is which and say why.
Score legal usability against the actual purposes. Line up your step-1 purpose list against N1–N3, purpose by purpose. The discipline: consent wording is a scope document, not a vibe — "assessment and settlement of the claim" covers some of your purposes and not others. Where a purpose lacks a basis, that is a RED with a named remediation (fresh notice/consent language, or a documented legitimate-basis analysis by the privacy office — the fix is theirs to own, yours to register). Note for GDPR-minded readers: the same purpose-limitation logic applies; write the row so it survives either regime.
Score ownership and retrievability. Ownership: a catalog row with no owner is not a formality gap — it is "nobody is accountable for the dataset your pilot depends on most." Retrievability: check each dataset's refresh cadence against your runtime demand; a nightly-batch source consumed at FNOL time means decisions on day-old data — decide whether that breaks the use case (for policy cover context: usually tolerable, document the staleness window) and score accordingly.
Build the prerequisite register. Every RED and AMBER becomes a row: Prerequisite | Blocking? | Owner (named person/team from the catalog or org) | Target date | Verification (how we will know it is done). Blocking means the pilot must not start; non-blocking means start with a documented workaround. Force yourself to name real owners from the landscape — "the business" owns nothing.
Pressure-test with the AI red-team. Prompt: "Here is my readiness scorecard and prerequisite register for this AI pilot. What readiness risk have I missed or underscored? Argue for the two most dangerous cells." The model may surface real gaps (eval-set consent, the pre-2023 consent_flags hole, the unowned dataset's PII classification) — adjudicate each against the evidence lines and update or rebut in writing.
Write the readiness verdict. Half a page: overall verdict (READY / READY-WITH-PREREQUISITES / NOT READY), the two pilot-killers named with their evidence lines, the prerequisite register attached, and the sentence the steering committee needs: what must be true, by whose hand, before day one. Readiness verdicts hedge when scorecards are thin — yours has citations, so it can afford to be blunt.
Legal usability is scored against the purposes you will actually pursue — including evaluation and improvement — not just the headline use case; consent collected for one purpose does not stretch to cover the interesting ones.
A complete lab has all of the following:
Data readiness is five questions asked before the pilot instead of during it — and the two that kill quietly are legal usability (does the basis cover every purpose, evals included) and quality on the one field the use case cannot function without. A scorecard with citations and named owners is what turns "subject to data readiness" from a delay into a plan.