Breaking-Change Upgrades and Language Migrations: Codemods for the Mechanical 80%, LLM for the Rest

Tier: Advanced · Workflow area: Refactoring & Migration

This playbook runs that hybrid for both cases — a breaking-change library upgrade driven by its changelog, and a language/typing migration slice (JS→TS, Java→Kotlin) — because they are the same playbook at different scales.

When to use

• •
  A dependency major-version bump with a real breaking-change list (Django LTS jumps, Spring Boot 2→3, React Router, SQLAlchemy 1.x→2.x).
• •
  A typed-language migration done in slices: JS→TS directory by directory, Java→Kotlin class by class.
• •
  Any change that is 80% repetitive edits across many files and 20% judgment calls.

Not for: upgrades with no breaking changes (just bump it) or rewrites where behavior is supposed to change (that is a feature project, not a migration).

Prerequisites

• •
  Green CI on the current version — a failing baseline makes "did the upgrade break this?" unanswerable. For poorly-tested target areas, build characterization tests first (pb-22).
• •
  The official changelog/upgrade guide saved locally (models trained before a release hallucinate its API; the changelog in-context overrides that).
• •
  Codemod tooling for your ecosystem: OpenRewrite (JVM), jscodeshift/ts-migrate (JS/TS), pyupgrade/Bowler (Python), plus any vendor-provided upgrade codemods.
• •
  An agent that can edit, build, and run tests in a loop (Claude Code, Copilot coding agent), on a dedicated branch.

The workflow

1. 1.

   Changelog → migration matrix. Feed the actual upgrade guide plus your repo's usage inventory:

   Copy

   grep -rn "from sqlalchemy" --include="*.py" src/ | cut -d: -f3 | sort | uniq -c | sort -rn > /tmp/usage.txt
   

   Copy

   Here is the official 1.4→2.0 migration guide [pasted] and our usage
   inventory [pasted]. Build a migration matrix: for each breaking change,
   (a) does it affect us — cite usage lines, (b) is the fix MECHANICAL
   (same rewrite everywhere) or JUDGMENT (depends on local semantics),
   (c) estimated blast radius in files. Mark changes that don't affect us
   as N/A with evidence. Flag anything in the guide you're uncertain about.
   

   The MECHANICAL/JUDGMENT split is the whole strategy: it decides what gets a codemod versus LLM attention.

2. 2.

   Run deterministic transforms for every MECHANICAL row. Prefer existing recipes (OpenRewrite's Spring Boot recipes, ts-migrate, vendor codemods); have the LLM write a codemod where none exists:

   Copy

   Write a jscodeshift codemod that rewrites `moment(x).format('YYYY-MM-DD')`
   to `formatISODate(x)` importing from '@/lib/dates', preserving all other
   moment usage. Include 5 test fixtures: 3 that must transform, 2 that must
   NOT (moment with other formats, moment in comments).
   

   Codemods are reviewable once and correct everywhere — an LLM editing 300 call sites freehand is 300 chances to hallucinate. Commit each codemod's output as its own commit for reviewability.

3. 3.

   Let the agent grind the compile/test loop on the residue. Now hand over the JUDGMENT rows:

   Copy

   Branch: upgrade/sqlalchemy-2. Codemods have handled [list]. Remaining
   breaking changes: [JUDGMENT rows from the matrix]. Work file by file:
   fix, then run `pytest -x --ff`; do not proceed past a failure. Rules:
   - Consult the migration guide (MIGRATION.md in repo root) before inventing
     an API — do not use methods not present in the guide or the installed
     package.
   - If a fix requires changing test assertions or behavior, STOP and list it
     for me instead of changing it.
   - Keep a log: file, change made, guide section relied on.
   

   The stop-rule is critical: an agent that "fixes" a failing test by editing the assertion has converted your upgrade into an unreviewed behavior change.

4. 4.

   Review the residue like a hostile PR, sample the mechanical. Review every JUDGMENT-row diff fully (this is where semantics live — e.g., SQLAlchemy 2.0's autocommit removal changing transaction boundaries, or a TS migration choosing any versus a real type). For codemod output, review the codemod plus a random 10-file sample per commit. For TS/Kotlin slices specifically: reject any/!! beyond an agreed budget — count them:

   Copy

   grep -rn ": any\|as any" --include="*.ts" src/migrated-slice/ | wc -l
   

5. 5.

   Verify beyond the unit suite, then ship in slices. Run integration tests, boot the app, and hit the golden paths — upgrade breakage loves runtime config and startup (deprecation warnings promoted to errors, changed defaults). Diff dependency trees (pip freeze | diff, mvn dependency:tree) for surprise transitive bumps. For migrations, merge slice-by-slice behind green CI rather than one mega-branch — Airbnb's TS migration ran file-set by file-set for months; mega-branches rot.

Tool variants

• Claude Code: best agentic loop discipline (hooks can enforce "run tests after every edit"); put the stop-rules in CLAUDE.md on the branch.
• Copilot coding agent: assign JUDGMENT rows as issues; it iterates against CI on its own PR — good fit for the file-by-file residue at $10/mo.
• OpenRewrite (JVM): for Spring/Java upgrades much of the matrix is pre-built as recipes; run recipes first, LLM only for what has no recipe; ts-migrate + jscodeshift play the same role for TS.

Verification

• •
  Full CI green, integration tests green, app boots with zero new deprecation warnings (or a triaged allowlist).
• •
  Every migration-matrix row is resolved: N/A-with-evidence, codemod commit, agent commit, or explicitly deferred.
• •
  No test assertions changed anywhere in the branch (enforce: git diff main...HEAD -- '**/test*' | grep '^-.*assert' reviewed line by line).
• •
  Agent log's cited guide sections spot-checked — the API calls it chose exist in the installed version.
• •
  For language slices: any/!! count within budget; new-language idioms verified on the sampled files.

Pitfalls

• LLM-editing what a codemod should own. Freehand edits across hundreds of sites is where hallucinated APIs and inconsistent rewrites breed. If the fix is describable as a rule, mechanize it.
• Trusting the model's memory of the new version. Models confidently produce the old API with new-version syntax, or invent transitional helpers. The changelog lives in-context, always.
• The agent "fixing" tests. Watch for assertion edits and deleted tests in every agent commit — it is the loop's way of lying to you.
• Big-bang branches. A 6-week upgrade branch conflicts with everything and reviews like a landslide. Slice by directory/module, merge continuously.
• Skipping the runtime check. Compile-green and unit-green miss config/startup breakage — boot it and click the golden paths before declaring victory.

Restricted-environment variant

• Deterministic-firstcodemods, OpenRewrite, ts-migrate, and the whole compile/test loop are local and policy-free; in restricted environments, push more work into the MECHANICAL column and shrink the LLM's residue.
• For the residue under client policysanctioned enterprise agent (Copilot Business, client-tenant Bedrock/Azure route) with the same stop-rules; or extract each JUDGMENT call-site into a sanitized snippet, resolve it in a chat model, and apply by hand.
• Local-model routegood enough to write codemods (small, self-contained task) even when not trusted to run the agentic residue loop — that split alone captures most of the value at $0.

Success criteria

• •
  The upgrade/migration lands sliced and green with every change traceable to a matrix row — no "and various other fixes" commits.
• •
  Mechanical work was done by reviewed transforms, not per-site freehand edits; residue diffs got full human review.
• •
  Zero behavior changes shipped undetected: assertions untouched, characterization suites (where present) green throughout.
• •
  The migration matrix + codemods are reusable by the next team doing the same upgrade — publish them internally.