No error, anywhere. Training runs clean, the API returns 200s, dashboards stay green. The only sign is slow and human: the monthly recall review shows the churn model missing customers it used to catch — recall drifted from 0.667 towards 0.5 over three weeks. Nobody can point to a failing anything, because nothing failed. The system is wrong, quietly, at full speed.
A shared repo in active development. A teammate tuned a feature and, mid-edit,
changed one divisor in the spend_share formula — / 3 became / 30. The code
still returns a perfectly legal float, so Python has nothing to raise, the
pipeline fits without complaint, and the 4.2 schema at the API door checks the
caller's input, not the server's own arithmetic. The repo had a tests/ folder,
and it was empty. Review missed the one-character diff among two hundred changed
lines. Linked modules: 4.5 (Testing ML code), 4.1 (Notebook to repo).
This is the signature ML failure: a wrong number instead of an exception, which
is worse than a crash because a crash stops the line while a wrong number ships.
Once the recall review raised the alarm, the trail was short: git log -p src/features.py listed every change to the feature code since the last good
month, and the divisor edit stood out in minutes. The expensive part was not
finding the bug — it was the twenty-one days of wrong scores already sent to the
field teams. The root cause is structural, not personal: no assertion anywhere
stated what spend_share must equal for a known customer, so no machine could
notice the moment the number changed. Any alarm that depends on a human staring
at a score is an alarm that takes leave.
Two moves, in order. First, correct the formula and retrain, confirming the
score returns to the recorded 0.824 accuracy and 0.667 recall. Second — the real
fix — pin every feature function with deterministic in-and-out unit tests: a
hand-worked row (income 24000, three orders at 600, spend_share exactly 0.025
under pytest.approx), the zero-orders edge case, and an input-never-modified
check. Run the new tests against the buggy commit to prove they catch it: the
E line shows 0.0025 against 0.025, and the ten-times-too-small gap names the
stray zero. From then on the same bug dies in the first pytest -q, 0.4
seconds after it is written.
Every feature function gets a worked-by-hand unit test before it is merged, and a pull request that touches feature code without touching its tests is sent back. Keep a model-quality gate on a frozen eval set as the second net: even a formula bug the unit tests somehow miss will drag recall below the floor and fail the gate. Treat "the code ran" as zero evidence of correctness in ML — only asserted numbers count. The full four-layer suite is module 4.5's drill.
Q: "Your model's quality degraded in production for three weeks and no system caught it. Walk me through what was missing." Strong answer: name the silent wrong number as a failure class distinct from crashes, explain why input validation at the API could not catch a server-side formula bug, and give the two nets — deterministic unit tests on feature functions and a threshold gate on a frozen eval set. Mentioning that you would prove the new tests catch the original bug by running them against the buggy commit signals real practice.