Build a Catalog Enrichment Agent with a Human Approval Queue

Tier: Advanced · Workflow area: Docs, lineage & governance

This playbook builds that loop — the emphasis is on the guardrails that make an autonomous enrichment agent trustworthy: read-only access, proposals-not-writes, evidence per proposal, and an approval gate.

When to reach for this

• •
  The catalog (Atlan, Unity Catalog, DataHub, OpenMetadata) is deployed but chronically under-filled, and manual campaigns keep decaying.
• •
  New assets appear weekly and you want enrichment to be continuous, not campaign-based.
• •
  You have already done a supervised bulk pass and trust the prompt/review loop enough to automate the drafting half.

Don't start here if you have never run a supervised bulk documentation pass — automate a workflow you've validated manually, not one you're inventing.

The workflow

1. 1.

   Inventory the gaps via the catalog API. Every serious catalog exposes assets and their metadata over REST/GraphQL. Pull assets where description IS NULL, owner is unset, or required tags are missing, and rank by usage (query count from warehouse logs) so the agent works on assets people actually touch.

2. 2.

   Assemble evidence per asset. For each gap, gather what a human documenter would look at: upstream lineage and upstream descriptions, the creating SQL or dbt model, top query patterns against the asset from query history (which columns get selected/joined/filtered), and most-frequent-querier teams (a strong ownership signal).

3. 3.

   Draft proposals with structured output. Each proposal carries the drafted value and its evidence:

   Copy

   import anthropic
   from pydantic import BaseModel
   
   class EnrichmentProposal(BaseModel):
       asset: str
       field: str               # description | owner | tag
       proposed_value: str
       evidence: list[str]      # citations into lineage/query-log evidence
       confidence: str          # high | medium | low
   
   class ProposalSet(BaseModel):
       proposals: list[EnrichmentProposal]
   
   client = anthropic.Anthropic()
   response = client.messages.parse(
       model="claude-opus-4-8",
       max_tokens=16000,
       thinking={"type": "adaptive"},
       messages=[{"role": "user", "content": build_evidence_prompt(asset_bundle)}],
       output_format=ProposalSet,
   )
   queue.extend(response.parsed_output.proposals)
   

   The prompt rules mirror pb-14: derive from evidence, never invent business rules, propose owner only when query-log concentration is decisive, emit low-confidence-with-question instead of guessing. Run nightly batches through the batch API — enrichment is the definition of a non-latency-sensitive workload.

4. 4.

   File into an approval queue, never write directly. Push proposals into the catalog's native suggestion/approval workflow if it has one (Atlan suggestions, DataHub proposals), else a lightweight review table or PR-based flow. The agent's credential is read-only on data and write-only to the proposal store — it must not be able to publish a tag or description itself. Approvals route by domain: proposals for finance-owned assets go to finance data owners.

5. 5.

   Close the loop with feedback. Track accept/edit/reject per proposal type. Rejections are prompt fuel: fold recurring correction patterns back into the drafting prompt ("owners are teams, never individuals"; "never tag exports as deprecated"). Accept-rate per confidence band also validates whether your confidence field means anything.

6. 6.

   Schedule and bound it. Run from your orchestrator (a nightly Airflow 3 DAG is fine) with hard bounds: max proposals per run (a reviewable number, e.g. 50), token budget per run, and an automatic pause if accept-rate drops below a floor — a degrading accept-rate means the agent is now generating review burden, not value.

Verify before you trust it

• •
  The agent's service principal provably cannot write catalog metadata directly — test by attempting a direct write with its credential.
• •
  Every proposal in the queue renders its evidence citations, and spot-checked citations actually support the proposed value.
• •
  Accept-rate is tracked per field type and confidence band; the pilot ran ≥2 weeks with accept-rate above your bar (e.g. 80% on descriptions) before scale-up.
• •
  Owner proposals were confirmed with the proposed owners for the first batches — mis-assigned ownership annoys people fast and burns the program's credibility.

Pitfalls

• Review-queue flooding. An agent that files 2,000 proposals on day one gets its queue ignored, permanently. Ranked, bounded batches keep approvals a five-minute daily habit.
• Ownership by query-volume alone. The heaviest querier is often a downstream consumer, not the owner. Require corroborating signals (creator, repo CODEOWNERS, schema naming) for owner proposals.
• Echo-loop descriptions. Once the agent's own approved descriptions enter the catalog, they become "evidence" for neighboring drafts. Tag agent-authored metadata so it never outweighs human-authored sources in prompts.
• Query logs as PII source. Query text can embed literal values (emails in WHERE clauses). Strip literals from query-history evidence before it enters prompts.
• Unbounded spend. Nightly LLM passes over a big estate add up. Per-run token budgets and batch-API pricing keep this a rounding error instead of a surprise.

Tools

• Claude API (batch + structured outputs)the drafting engine; batch pricing fits the nightly cadence, structured output feeds the queue without parsing glue.
• Catalog APIs (Atlan / Unity Catalog / DataHub / OpenMetadata)gap inventory in, proposals out; use the native suggestion workflow where one exists.
• Airflow 3 (or Dagster)scheduling, bounds, and observability for the agent as a plain orchestrated pipeline.

Client-safe variant

For a client's catalog, run the agent inside the client's environment with client-issued read-only credentials, and route the LLM calls through whatever provider arrangement the client has approved (their cloud's model endpoint, or self-hosted). Query-history evidence is the risky input — get explicit sign-off that query text may be processed, and always strip literals. Contract the deliverable as the system plus its metrics (accept-rate dashboard, coverage trend), not just enriched metadata: the accept-rate history is what lets the client trust and continue running the agent after handover.