A refactor accidentally dropped the three engineered features from training, the model was retrained, and a clearly worse model — recall down from 0.667 to 0.487, one churner in two now missed — shipped to production. The test suite ran before the release and printed all green. When the quality drop finally surfaced through complaints, the first question was "how did fourteen tests pass?", and the answer was uncomfortable: they were designed to.
A repo whose suite grew for coverage numbers rather than protection. The tests
called load_customers(), add_features(), pipe.fit(), pipe.predict() and
the API endpoints — and asserted only survival: fit completed, predict
returned something, the response code was 200. Not one assert stated a value:
no hand-worked feature number, no probability range, no score floor. The suite
measured "does it run", never "is it right" — and a worse model runs perfectly.
Linked modules: 4.5 (Testing ML code), 4.2 (Serving with FastAPI).
Audit a suite by reading its asserts, not its count: every assert here checked survival, none checked a number, so the suite could not fail for any quality reason. The definitive audit is a mutation drill — plant a bug that must be caught and watch the board: with the divisor typo planted, fourteen green; with a worse model swapped in, fourteen green. A suite that stays green under planted damage is measuring nothing. The root confusion is treating a smoke test's question ("does the service answer?") as if it answered the gate's question ("is the model still good enough?"). Smoke tests cap a suite; they cannot carry it.
Add the missing layer rather than rewriting everything: a model-quality gate that loads the shipped joblib and holds it to agreed floors — recall at least 0.60, accuracy at least 0.78 — on a frozen eval set, the same seeded split every run so a change in score can only come from the model. Rerun the mutation drill to prove the cure: the worse model now fails exactly two tests, with the evidence in the E lines (0.487 against 0.60). Then upgrade the strongest of the survival tests into value tests — hand-worked feature numbers, probability ranges, save/load predictions preserved — and keep the smoke tests as what they are: the cap, not the gate.
For every new test, ask the one-line review question: "what planted bug turns this red?" — a test with no answer is decoration and should not merge. Run the regression-catch drill from module 4.5 whenever the suite grows: plant a formula typo, a schema change and a worse model, and require that each turns the right layer red. Track the gate's floors in the repo where diffs are reviewed, so lowering a floor is a visible, argued decision — not a quiet edit the night before a release.
Q: "Your team has 95% test coverage and still shipped a model that was much worse. How is that possible, and what do you change?" Strong answer: coverage counts lines executed, not promises checked — a suite asserting survival passes any worse model. Name the mutation drill as the honest audit, and the fix as a quality gate with numeric floors on a frozen eval set. The senior signal is separating the smoke test's question from the gate's question, and asserting on numbers, not on the absence of exceptions.