Lab 08 · Migration Decomposition Harness

The skill is not the logging swap; it is designing decomposition boundaries and verification so that agent output is trustworthy at a scale where you cannot review line-by-line.

Time & tier

~2 hours hands-on · Advanced.

Setup

Deterministic inline generator — no AI needed to build the scaffold. In a fresh git repo, save and run python make_scaffold.py:

# make_scaffold.py
import os

os.makedirs("app", exist_ok=True)
open("app/__init__.py", "w").close()
with open("app/legacy_log.py", "w") as f:
    f.write("def log(msg):\n    print('[LEGACY] ' + msg)\n")

FORMS = [
    ['print("[{n}] processed %d records" % count)'],
    ['import sys',
     'sys.stderr.write("{n}: retry attempt " + str(count) + "\\n")'],
    ['from app.legacy_log import log',
     'log("{n} finished with " + str(count))'],
    ['print(f"[{n}] skipped {{count}} stale rows")'],
]
for i in range(40):
    n = f"task{i:02d}"
    body = "\n".join("    " + ln.format(n=n) for ln in FORMS[i % 4])
    with open(f"app/{n}.py", "w") as f:
        f.write(f'def run(count: int) -> int:\n'
                f'    """Process {i} widgets."""\n'
                f'    result = count * {i + 1}\n'
                f'{body}\n'
                f'    return result\n')
print("wrote 40 modules")

And the deterministic checker, check.py:

import importlib, pathlib, re, sys

BANNED = [r"\bprint\(", r"sys\.stderr\.write", r"legacy_log"]
fails = []
for p in sorted(pathlib.Path("app").glob("task*.py")):
    src = p.read_text()
    for pat in BANNED:
        if re.search(pat, src):
            fails.append(f"{p}: banned pattern {pat}")
    if "logging.getLogger(__name__)" not in src:
        fails.append(f"{p}: no module logger")
    if re.search(r'(%\s*\(|%\s*count|"\s*\+|f"\[)', src):
        fails.append(f"{p}: eager string formatting in log call")
    try:
        m = importlib.import_module(f"app.{p.stem}")
        expected = int(p.stem[4:])
        if m.run(3) != 3 * (expected + 1):
            fails.append(f"{p}: behaviour changed")
    except Exception as e:
        fails.append(f"{p}: import/run failed: {e}")
print("\n".join(fails) or "ALL CLEAN: 40/40")
sys.exit(1 if fails else 0)

Target state per module: logger = logging.getLogger(__name__) at module level, the legacy call replaced by a logger.info(...) with lazy args (logger.info("[%s] processed %d records", name, count)), return values untouched, app/legacy_log.py deleted last. Scaffold verification: ls app/task*.py | wc -l = 40; python check.py exits 1 listing ~80+ violations; git add -A && git commit -m baseline.

Guided steps

1. 1Write the decomposition spec (harness/plan.md) before touching an agent. Decide: batch size and boundary (by file range? by legacy-pattern form? — decision point: pattern-based batches give each agent one transformation to learn, file-range batches parallelize more evenly; justify your pick in two sentences), the exact per-module transformation contract (input pattern → output pattern, with one worked example per legacy form), and the definition of done per batch: python check.py scoped to the batch passes.
2. 2Make the checker batch-scoped. Add sys.argv support so python check.py app/task00.py app/task09.py ... (or a --range 0 9 flag) checks only that batch. This 5-minute edit is what makes parallel verification possible — agents get fast, scoped feedback.
3. 3Author one reusable task prompt (harness/task-prompt.md): the transformation contract, the worked examples, the banned patterns, "change nothing else — no docstring edits, no reformatting", and the command the agent must run to self-verify. One prompt, parameterized by file list — never hand-write per-batch prompts.
4. 4Run batches in parallel. Free-tier routes: Copilot agent mode on one batch per session; multiple Windsurf/Gemini chats with pasted file contents; or if you have any CLI agent, one git worktree per batch (git worktree add ../batch-b main) so edits can't collide. Sequential batches through one tool is an acceptable fallback — the harness, not the concurrency, is what's graded. Keep per-batch transcripts.
5. 5Build the aggregator (harness/aggregate.sh): for each batch, apply/merge its diff, run the scoped check, run the full python check.py, and append a line to harness/verification.log — batch B: files=N scoped=PASS full=PASS/FAIL sha=COMMIT_SHA. A batch that fails scoped verification gets bounced back to its agent with the checker output, not hand-fixed; log the bounce. Decision point: merge order — batches touching legacy_log importers must land before deleting legacy_log.py itself.
6. 6Aggregate and finish. When all batches merge and python check.py prints ALL CLEAN: 40/40, delete app/legacy_log.py, re-run, commit. Produce the human-review artifact: git diff baseline..HEAD --stat plus a 10-line sampled review (you eyeball 4 random modules, one per legacy form — record which).
7. 7Post-mortem paragraph in plan.md: which batch bounced and why, what you'd change in the task prompt for 400 files instead of 40.

Verification

• •
  python check.py exits 0 with ALL CLEAN: 40/40; grep -rn "print(\|legacy_log\|stderr" app/ returns nothing.
• •
  Behaviour preserved: the checker's run(3) assertions all pass (this is your characterization layer — it ran on every batch).
• •
  harness/verification.log shows one line per batch, at least one scoped check per batch, and any bounces with their reason.
• •
  The final diff touches only app/task*.py and deletes legacy_log.py — git diff baseline..HEAD --stat shows no collateral files.
• •
  Honesty check: at least one batch should have failed a check at some point (agents reliably eager-format or edit docstrings); if your log shows zero bounces, confirm your checker actually catches a planted violation — break one file by hand, run the aggregator, watch it fail, revert.

Completion artifact

Keep the harness/ directory (plan.md, task-prompt.md, aggregate.sh, verification.log) plus the final aggregated diff. Self-grading rubric:

• Decomposition design (0–2): batch boundaries justified, merge-order dependency identified = 2.
• Verification engineering (0–3): scoped checker works, aggregator logs every batch, planted-violation test performed = 3.
• Prompt reuse (0–2): one parameterized task prompt, no per-batch rewrites = 2.
• Outcome (0–2): 40/40 clean, behaviour preserved, zero collateral changes = 2.
• Scale reflection (0–1): post-mortem names one concrete change for 10x scale = 1.

Pass = 7/10.

Stuck?

• Agents keep "improving" docstrings or type hints. Tighten the contract with a negative example in task-prompt.md and rely on the aggregator: any diff line outside the log-call region fails the batch. Bounce, don't hand-fix — hand-fixing at scale is the failure mode this lab exists to unlearn.
• The eager-formatting regex flags your correct code. The checker's heuristic is crude by design (it's yours — improve it). Lazy logger.info("... %s", x) with no + or f-string inside the call passes; if a legitimate f-string elsewhere in a module trips it, scope the regex to lines containing logger..
• Worktrees feel heavy. Skip them: batches touching disjoint files can't conflict, so sequential agent sessions committing to one branch works — keep the per-batch commits separate so the aggregator can attribute failures.

Client-safe variant

The harness pattern is the most client-transferable skill in this track because verification is fully local and deterministic: check.py-style gates ship no code to any model. On restricted engagements, run the transformation itself with the sanctioned agent (Copilot Business) or a local model — mechanical pattern swaps are exactly where local coder models are production-usable — or with a pure codemod (libcst/jscodeshift) generated once from a redacted example, keeping the LLM out of the loop entirely. The verification.log becomes your client deliverable: evidence that every machine change passed a machine gate plus sampled human review.

Answer key (don't read before attempting)

No hidden seeds — the scaffold's traps are structural, and here they are for post-lab comparison: (1) four distinct legacy forms means single-example prompts under-generalize (agents nail print and miss the sys.stderr.write inline-import form, leaving a dead import sys behind — did your checker or your sampling catch dead imports? The provided checker does not: that gap is deliberate); (2) form 3 creates the dependency trap — modules importing legacy_log must migrate before the helper is deleted, or intermediate states break imports for every batch after; (3) form 4's f-string is the eager-formatting bait — a naive migration to logger.info(f"...") preserves the anti-pattern the migration exists to remove; (4) the run(3) return-value check is the behaviour pin — agents that "clean up" result = count * k get caught by the checker, which is the whole argument for verification-first decomposition.