Contracts, invoices, scanned reports, and slide decks hold analytical data that never reaches a table.
Tier: Core · Workflow area: Document Intelligence & Multimodal Extraction
This playbook converts heterogeneous documents into a clean, typed dataset: parse layout-aware where possible, fall back to vision models where parsing fails (scans, charts, complex tables), extract against a Pydantic schema, and attach an eval sample so you can state the error rate of the resulting table.
Skip vision models when documents are born-digital with clean text layers — a parser plus text extraction (pb-14) is cheaper and more accurate.
docling or unstructured for parsing; pypdf/pdfplumber for triage; the pb-14 extraction stack.Triage the corpus. Classify each document programmatically: has text layer? page count? contains tables/charts? scanned? Route accordingly — text-parse route, vision route, or hybrid. Log the mix; it drives cost.
Parse route (born-digital docs). Docling converts PDF → structured Markdown preserving tables and headings:
from docling.document_converter import DocumentConverter
md = DocumentConverter().convert("contract.pdf").document.export_to_markdown()
Feed the Markdown (or the relevant sections — split on headings for long docs) into the pb-14 extraction function with your target schema.
Vision route (scans, charts, hostile layouts). Send the document or page images directly; current APIs take PDFs natively:
import base64
pdf_b64 = base64.standard_b64encode(open("scan.pdf", "rb").read()).decode()
resp = client.messages.parse(
model="claude-opus-4-8", max_tokens=2048,
messages=[{"role": "user", "content": [
{"type": "document", "source": {"type": "base64",
"media_type": "application/pdf", "data": pdf_b64}},
{"type": "text", "text": "Extract the fields defined by the schema."}]}],
output_format=InvoiceSchema)
For chart extraction, add schema fields for axis labels, units, and per-series data points, plus read_confidence: Literal["exact","approximate"] so downstream analysis knows which numbers were eyeballed off a bar chart.
Schema design for documents. Beyond pb-14 basics: include source_page: int and a short evidence_quote: str per critical field — verbatim quotes make human verification 10x faster and hallucinations obvious (a quote that isn't in the document is an automatic reject).
Two-pass verification for high-stakes fields. Extract, then run a second cheap-tier call: "Given this document and this extracted value with its quote, confirm or correct." Disagreements go to human review. This catches most silent misreads for a few percent extra cost.
Evaluate against gold, then run the corpus. Field-level accuracy on the 20 gold documents, broken down by route (parse vs vision) and field. Batch the full corpus via pb-16's pipeline; sample 2% for ongoing QA.
Numeric fields from vision extraction (totals, dates, IDs) are the highest-risk outputs in this entire track — a transposed digit in a contract value is worse than a missing value. Always cross-check derived arithmetic (line items sum to total?) as schema-level validators.
file_id for reuse across many questions on one doc); ≤32 MB per request. Pair with messages.parse for typed output.client.responses.parse with input_file/image content parts and text_format=Schema.null = "not present in document" from extraction failure; add an explicit field_present: bool where absence is meaningful (e.g. missing termination clause).source_page, quote), and a stated field-level error rate.