Generate Unit and Integration Tests with a Coding Agent — Then Prove Them with Mutation Testing

Tier: Core · Workflow area: Automation engineering

This playbook pairs the fastest use of coding agents — backfilling tests for untested code — with the verification that makes the output trustworthy. Generated coverage without assertion strength is a liability that looks like an asset, which is the most dangerous kind.

When to use

• Legacy modules with no tests that you must change soon — tests first, refactor second.
• AI-authored application code arriving in PRsQA validating dev-side generated code needs targeted tests fast (the pb-49 territory, executed with this recipe).
• Coverage mandatesyou are required to hit a number; this is how the number stays meaningful.

When NOT to use:

• Code about to be deleted or rewrittencharacterization tests for a module with a fortnight to live are waste; test the replacement.
• Untestable-as-designed codeif everything is static calls and hidden I/O, refactor seams first or the agent will mock the world into meaninglessness.

Prerequisites

• A coding agent with test-run permission in the repo — the write-run-fix loop is the whole speed advantage.
• Mutation tooling installedmutmut for Python, Stryker for JS/TS, PIT for JVM — and runnable on the target module in minutes, not hours.
• A statement of intent for the module — docs, ADRs, or a product-owner conversation; without intent the tests can only enshrine the code.

The recipe

1. 1.

   Pick the target and capture current behavior intent. Tell the agent what the module is for — without intent, it writes tautologies that enshrine current behavior, bugs included:

   Copy

   Generate pytest tests for src/billing/proration.py (currently untested).
   Purpose: computes mid-cycle subscription charges; must round half-up to
   cents; must never produce negative charges; upgrades take effect
   immediately, downgrades at period end.
   Requirements:
   - Test observable behavior via public functions only.
   - Include boundary cases: first/last second of period, zero-day
     remainder, leap day, plan price of 0.
   - Mark anything that looks like a BUG in the current code with a
     "# SUSPECTED BUG" comment and test the *correct* behavior as xfail —
     do not lock in the wrong behavior silently.
   - No mocking of pure functions. Use freezegun for time.
   Run: pytest tests/billing/test_proration.py -q — iterate until green.
   

2. 2.

   Review the suspected-bug flags first. This is the highest-value output of the whole generation: places where written intent disagrees with the code. Each one goes to a human decision — real bug (file it, keep the xfail) or wrong intent (fix the test).

3. 3.

   Check coverage — as a floor, not a verdict:

   Copy

   pytest tests/billing/ --cov=src/billing --cov-report=term-missing -q
   

   Uncovered lines are work remaining. Covered lines prove nothing yet.

4. 4.

   Run the mutation gate. Python — mutmut; JS/TS — Stryker:

   Copy

   mutmut run --paths-to-mutate src/billing/proration.py
   mutmut results
   

   Copy

   npx stryker run --mutate "src/billing/**/*.ts"
   

   Mutation testing changes >= to >, deletes statements, flips constants — each surviving mutant is a code change your suite cannot detect. Expect agent-generated suites to land far below their coverage number on first run; that gap is the finding.

5. 5.

   Feed survivors back to the agent — this loop is the playbook:

   Copy

   mutmut reports these surviving mutants in proration.py: [paste
   `mutmut show` output for each]. For each survivor, either strengthen an
   existing test or add one that kills it. Do NOT write assertions that
   merely restate the mutated line; assert the behavioral consequence.
   Rerun pytest and tell me which mutants you expect each change to kill.
   

   Re-run mutmut. Two or three iterations typically move a suite from ~55% to 85%+ mutation score. Survivors that remain should be explainable — equivalent mutants or accepted risk, listed in the PR description.

6. 6.

   Human-review the final suite against the pb-13 rubric — especially over-mocking and tests that duplicate each other's kills — then merge with both numbers in the PR: coverage and mutation score.

Make it yours

• Jest/Vitest + Stryker (TS/JS)identical loop; configure stryker.config.json with "testRunner": "vitest" and use --incremental so iteration runs stay fast on big modules.
• JVM teamsPIT (pitest) via Maven/Gradle plugs into the same feedback loop; agents handle JUnit 5 + AssertJ generation well when you name those libraries in the prompt.
• Integration-tier variantfor repository/service tests against a real database (Testcontainers), skip mutation testing's cost on slow tests: gate instead with fault injection — have the agent add tests for connection failure, constraint violation, and duplicate-key paths, and verify each by inducing the fault once.

Quality check

• Mutation score ≥80% on the target module, with every surviving mutant classified (equivalent / accepted / TODO) in writing.
• All SUSPECTED BUG flags dispositioned by a humaneach is now a filed defect with an xfail test, or a corrected test with the intent note updated.
• No behavior-free testsnothing that asserts a mock was called with what the mock was configured for; delete on sight.
• Time and randomness are controlledfrozen clocks and seeded RNG; agent tests that pass at authoring time and fail at month-end are a classic.
• The suite runs fast enough to keepbackfilled unit tests exceeding a few seconds total will get skipped by humans; push slow paths to the integration tier deliberately.
• Both numbers are in the PRcoverage and mutation score, so the reviewer approves detection power, not line-execution theater.