Turn Column-Level Lineage into an Impact-Analysis Brief

Tier: Core · Workflow area: Docs, lineage & governance

Lineage tools give you the graph; they do not give you the judgement or the write-up. This playbook feeds column-level lineage plus a proposed change to an LLM and gets back an affected-consumers brief and a migration checklist a human can verify and send.

When to reach for this

• •
  Renaming, retyping, or dropping a column in a model with more than a handful of downstream consumers.
• •
  Deprecating a model or source and needing a consumer-by-consumer migration plan.
• •
  A stakeholder asks "what breaks if we change X?" and you need a defensible answer today, not after a manual graph walk.

For a column with one downstream reference, just grep — the brief costs more than the fix.

The workflow

1. 1.

   Export the lineage, don't describe it. The LLM must reason over the real graph, not your memory of it. From dbt:

   Copy

   # Direct + transitive children of the model being changed
   dbt ls --select fct_orders+ --output json > /tmp/downstream.json
   

   For column-level precision, export from your catalog or observability tool (Monte Carlo, Atlan, dbt Explorer) — column lineage is what separates "37 downstream models" from "4 models that actually select this column".

2. 2.

   State the change precisely, then prompt for the brief.

   Copy

   Proposed change: in model fct_orders, rename column `order_total` to
   `order_total_gross_usd` and change type from FLOAT to NUMERIC(18,2).
   
   Attached: (1) column-level lineage export for fct_orders.order_total,
   (2) the model SQL, (3) list of BI dashboards mapped to downstream models.
   
   Produce an impact-analysis brief with these sections:
   1. Directly affected consumers — models/dashboards that reference the column
      BY NAME, each with the exact file/line or dashboard field.
   2. Indirectly affected — consumers of derived columns; explain the derivation chain.
   3. Not affected — downstream nodes in the lineage that do NOT touch this column
      (so reviewers can confirm your reading of the graph).
   4. Breakage type per consumer: hard failure, silent semantic change, or cosmetic.
   5. Migration checklist in dependency order, with a backwards-compatible
      transition step (view alias or dual-write) if any consumer cannot deploy same-day.
   Only cite consumers present in the attached lineage. If the lineage seems
   incomplete (e.g. dashboards not mapped), say so explicitly.
   

   Expected output: a structured brief where every claimed consumer has a citation into your export, plus a checklist ending with "remove the compatibility alias".

3. 3.

   Verify the two failure directions. The brief can be wrong two ways: consumers it invented, and consumers it missed. Check invented ones by grepping each citation (grep -rn "order_total" models/ | grep -v order_total_gross); check missed ones by diffing the brief's consumer list against the raw lineage export line count. The "Not affected" section exists precisely so you can spot-check the model's reading of the graph.

4. 4.

   Dry-run the change on a branch. Apply the rename with a compatibility view, run dbt build --select fct_orders+ (state-based selection in CI is even better), and compare failures against the brief's "hard failure" list. Every mismatch is either a lineage gap or a reasoning error — both worth knowing before you send the brief.

5. 5.

   Send the brief, keep the checklist. The consumer-facing sections go to affected teams; the migration checklist becomes the PR description. Because each step cites real files, reviewers can approve on evidence rather than trust.

Verify before you trust it

• •
  Every consumer in the brief maps to a line in the lineage export or a grep hit — no citation, no claim.
• •
  The dry-run failure set matches the brief's hard-failure list exactly.
• •
  Silent-semantic-change cases (type narrowing, rounding) are validated with a before/after query on real data, not reasoning alone: SELECT count(*) FROM x WHERE order_total != CAST(order_total AS NUMERIC(18,2)).
• •
  BI-layer impact is confirmed in the BI tool's own usage/lineage view — dashboard mappings are the least reliable part of most lineage graphs.

Pitfalls

• Table-level lineage masquerading as column-level. If your export is table-grained, the LLM will over-report impact by an order of magnitude. Say which grain you supplied and make the brief say which grain it assumed.
• Lineage blind spots. Ad-hoc queries, notebooks, reverse-ETL syncs, and spreadsheet exports rarely appear in the graph. The brief should carry a standing caveat listing what the lineage cannot see.
• Confusing "references" with "breaks". A downstream SELECT * won't fail on a rename but silently changes its output schema. Insist on the breakage-type classification per consumer.
• Stale exports. Lineage from last month's manifest describes last month's project. Regenerate the export in the same session as the brief.
• Treating the brief as the approval. It is an input to the change process, not a sign-off. The dry run is the sign-off.

Tools

• dbt lineage (dbt ls / manifest / dbt Explorer)free, in-repo source of model and column lineage for dbt projects.
• Claude (or another frontier LLM)the synthesis layer that turns graph + change into a consumer brief; needs the export pasted or attached, not summarized.
• Monte Carlo / Elementaryend-to-end lineage including BI-layer mapping; Elementary is the OSS path when there's no Monte Carlo budget.

Client-safe variant

In client engagements, lineage exports often reveal the client's entire data estate — treat the export itself as confidential data. Strip dashboard names and business terms that identify the client before sending anything to an external API, or run the synthesis step on an approved/self-hosted model and keep the raw export inside the client's environment. Deliver the brief on client letterhead conventions (their template, their ticket system) and attach the dry-run evidence; consultancies that show the failure-list-matches-brief verification step convert this from "AI wrote a doc" into an auditable impact assessment the client can reuse.