AI can take a legacy module from 20% to 80% coverage in an afternoon — and most of those generated tests will be worthless assertions like assert result is not None. Coverage percentage measures lines executed, not bugs caught.
Tier: Foundation · Workflow area: Testing & Quality
This playbook pairs AI test generation with mutation testing, so the metric you report is "mutants killed," not "lines touched." That is the difference between a suite that protects a refactor and one that decorates a dashboard.
Skip this for code scheduled for deletion, or for modules where integration tests already exercise the real risk (I/O boundaries, third-party calls) — unit-coverage theater there wastes the mutation budget.
coverage.py, nyc, JaCoCo).mutmut or cosmic-ray (Python), Stryker (JS/TS/C#), Pitest (JVM). All have free tiers or are OSS.Map the risk, not the gaps. Run coverage and ask the assistant: Here is the coverage report and the module source. Rank the uncovered functions by blast radius: which are called from the most places, mutate state, or handle money/auth/dates? Give me a top-5 target list with one sentence of risk each. You test the dangerous 5, not the alphabetical 40.
Generate behavior-first tests per target. For each target function, prompt:
Write pytest tests for `apply_late_fee()` below. Rules:
- One behavior per test, named test_<behavior>, e.g. test_waives_fee_within_grace_period.
- Assert on concrete output values and state changes, never just "is not None" or "no exception".
- Cover: the happy path, each branch, boundary values (day 0, day 30, day 31 of grace),
and the documented error path (negative balance raises ValueError).
- Use the existing fixtures in tests/conftest.py; do not invent new database mocks.
Curate before you commit. Delete tests that restate the implementation (mocking the internals and asserting the mock was called), tests asserting incidental behavior (dict ordering, exact log strings), and duplicates. Expect to cut 20–40% of what was generated. A test you would not want to maintain is a liability, not coverage.
Run the mutation gate. mutmut run --paths-to-mutate src/billing/fees.py. Every surviving mutant is a concrete gap: the mutation tester changed >= to > or deleted a line, and your suite stayed green.
Close survivors with targeted prompts. Paste survivors back: Mutant 14 survived: line 52, days >= gracebecamedays > grace. Write the single test that kills exactly this mutant. Repeat until the mutation score on the target functions clears your threshold (70%+ is a strong bar for legacy code).
Commit with the real metric. PR description states both numbers: "Coverage 22% → 81%; mutation score on changed files 76% (mutmut report attached)."
Coverage says your tests ran the code; mutation score says your tests would notice if the code were wrong. Only the second one justifies trusting the suite during a refactor.
Client contract forbids sending source to external APIs? Use the client-sanctioned enterprise assistant (Copilot Business with the no-training flag, or Bedrock/Azure OpenAI inside the client tenancy). If none exists: extract the function signature, docstring, and branch structure into a sanitized skeleton with neutral names, generate tests against the skeleton, then rename locally. Local fallback — Qwen-Coder-class models via Ollama handle single-function test generation adequately; keep the curate-and-mutate steps identical, since they run entirely on your machine.
Test fixtures often contain production-shaped data. Never paste real customer records into a hosted tool to "make the fixtures realistic" — see pb-29 for synthetic data that respects your schema.