Lab 09 · Coverage Lift With a Mutation-Testing Quality Gate

Time & tier

Core · ~75 minutes hands-on.

Setup

Everything is inline — no repo to clone. You need Python 3.11+, a coding assistant on a free tier (Copilot Free, Gemini Code Assist, Windsurf, or Continue.dev with any key), and three OSS packages:

mkdir lab09 && cd lab09 && python -m venv .venv && source .venv/bin/activate
pip install pytest coverage "mutmut>=3.0"
mkdir tests

Create pricing.py with this exact content — it is deliberately gnarly:

from datetime import date

TIERS = [(0, 0.0), (10, 0.05), (50, 0.10), (100, 0.18)]

def volume_discount(qty: int) -> float:
    if qty < 0:
        raise ValueError("negative qty")
    rate = 0.0
    for threshold, r in TIERS:
        if qty >= threshold:
            rate = r
    return rate

def apply_coupon(subtotal: float, code: str | None, today: date) -> float:
    if not code:
        return subtotal
    code = code.strip().upper()
    if code == "WELCOME10":
        return subtotal * 0.90
    if code.startswith("FLAT"):
        try:
            amount = int(code[4:])
        except ValueError:
            return subtotal
        return max(subtotal - amount, 0.0)
    if code == "EOFY" and today.month == 3 and today.day >= 25:
        return subtotal * 0.85
    return subtotal

def total(unit_price: float, qty: int, code: str | None = None,
          today: date | None = None) -> float:
    today = today or date.today()
    subtotal = unit_price * qty
    subtotal *= (1 - volume_discount(qty))
    subtotal = apply_coupon(subtotal, code, today)
    return round(subtotal + 0.0, 2)

Create setup.cfg so mutmut knows what to mutate (if your mutmut version reads config differently, check mutmut --help):

[mutmut]
paths_to_mutate=pricing.py
tests_dir=tests/

Guided steps

1. 1.

   Baseline. Run coverage run -m pytest && coverage report -m — with no tests you're at 0%. Note this number; your artifact needs before/after.

2. 2.

   Generate a first test batch. Give your assistant this prompt, pasting the full module:

   Copy

   Write pytest tests for this module. Rules: every test must assert an exact
   expected value computed by hand from the code's logic — no snapshotting the
   function's own output back into the assertion. Cover: tier boundaries (9/10,
   49/50, 99/100 units), negative qty, coupon whitespace/case handling, FLAT
   coupons including malformed ones and ones larger than the subtotal, the EOFY
   date window edges, and rounding to 2 decimals. Use freeze-free style: pass
   `today=` explicitly, never rely on date.today().
   

3. 3.

   Curate before running. Read every generated test and delete or fix any that (a) assert the function's own output ("characterization by accident"), (b) omit today= on EOFY tests, or (c) test nothing (call without assert). This curation pass is the lab.

4. 4.

   Hit the coverage bar. Run coverage run -m pytest && coverage report -m. Iterate with the assistant on uncovered lines (-m shows them) until ≥80%. Prompt pattern: "Lines 18–21 of apply_coupon are uncovered; write tests that reach them, with hand-computed expected values."

5. 5.

   Run the mutation gate. mutmut run then mutmut results. Mutmut mutates operators and constants (>= → >, 0.90 → 0.91) and re-runs your suite; a mutant that passes your tests "survives" and exposes a weak assertion.

6. 6.

   Kill survivors deliberately. For each survivor, run mutmut show <id>, decide whether it is (a) a weak assertion — strengthen the test, (b) an untested boundary — add a test, or (c) an equivalent mutant — document why. Ask the AI to draft the killing test, but you name the boundary. Target: ≥80% of mutants killed.

7. 7.

   Write the one-paragraph verdict. In NOTES.md, record: initial coverage, final coverage, mutants total/killed/survived, and the two most interesting survivors with what they revealed.

Verification

• •
  coverage report shows ≥80% for pricing.py.
• •
  mutmut results shows ≥80% killed, and every survivor has a written disposition in NOTES.md.
• •
  At least three tests pin the exact tier boundaries (qty 10, 50, 100) — grep your tests for these literals.
• •
  Deleting any single assertion at random and re-running mutmut produces new survivors (spot-check once) — proving assertions carry weight.

Completion artifact

A lab09/ folder containing pricing.py, tests/, NOTES.md, and the mutmut output. Self-grade (3 points = pass at 2+):

1. 1.
   Coverage ≥80% AND mutation kill rate ≥80%.
2. 2.
   NOTES.md correctly classifies every survivor (weak assertion / missing boundary / equivalent).
3. 3.
   No test in the suite asserts a value produced by calling the function under test to compute the expectation.

Stuck?

• Mutmut errors on config: mutmut 3.x changed its CLI; run mutmut run --help and move config to whichever of setup.cfg/pyproject.toml it documents. As a fallback, pip install "mutmut<3" and run mutmut run --paths-to-mutate pricing.py --tests-dir tests/.
• AI keeps writing self-referential asserts: add "compute expected values by hand and show the arithmetic in a comment" to the prompt.
• Coverage stuck below 80%: the malformed-FLAT branch (except ValueError) and qty < 0 raise are the usual misses.
• Everything survives: your tests probably assert result is not None. Assert exact floats.

Client-safe variant

On a client codebase where source cannot leave the machine: use the client-sanctioned assistant (Copilot Business, Amazon Q, or a local model via Ollama + Continue.dev — comprehension and test-generation quality is adequate for this task). If no assistant is permitted at all, paste only function signatures plus your own plain-English behavior notes into a general LLM to get test skeletons, then fill in expected values locally. The mutation-gate discipline is tool-independent and is the transferable half of this lab.

Answer key (don't read before attempting)

Seeded weaknesses your suite should expose: (1) tier boundaries are inclusive — qty=10 gets 5%, a >=→> mutant survives unless you test 10/50/100 exactly; (2) WELCOME10 applies after volume discount — order matters, test a case where both apply; (3) FLAT parses code[4:] so FLAT050 works but FLAT 50 doesn't — most AI suites miss the space case; (4) EOFY window is March 25–31 only — test March 24 and April 1 as negatives; (5) max(..., 0.0) means FLAT coupons can zero but never go negative; (6) round(x + 0.0, 2) uses banker's rounding — round(2.675, 2) is 2.67, a classic surviving-mutant site; (7) volume_discount(-1) must raise, and mutants deleting the raise survive if untested.