A model scores suspiciously well — 0.93, 0.97, 0.99 — and instead of doubting it, the team explains it. The importance chart is crisp, the SHAP stories are beautiful, one feature dominates every local explanation, and the deck almost writes itself. In production the model collapses, and the post-mortem finds the star feature was information from after the outcome — a leak the explanation had been faithfully describing all along.
Churn or default prediction on a warehouse table. A column like
winback_voucher_sent, refund_processed or account_frozen sits in the
snapshot — written by an operations process that runs AFTER the outcome the model
predicts. Module 1.7's time-of-knowledge audit was skipped, and the near-perfect
score was greeted as good news. The explanation tools then did their job
perfectly: they showed, in detail, exactly how the model was cheating.
Linked modules: 1.7 (leakage discipline) and 1.11 (honesty rules).
Explanations describe what a model uses, never whether it should. Every tool in the interpretation kit — impurity, permutation, partial dependence, SHAP — will rank a leaked column as supremely important, because to the model it IS supremely important. In the 1.11 exercise version: leaked accuracy 0.93 against an honest 0.82, the voucher flag holding 0.58 of impurity credit, topping permutation at +0.21, and partial dependence stepping from 0.05 to 0.59 on that one flag. None of those numbers is a malfunction; all of them are the leak, measured accurately. Confirm with 1.7's question, asked of the top feature: on the morning we score a new customer, does this value exist yet? If the answer is no, or "only after the outcome", the explanation was a confession.
Reverse the order of work: suspicious-score-first, explain-second. Any score that would beat your baseline by a miracle margin triggers a leakage audit BEFORE any explanation work — walk the top features and ask, for each, when the value gets written and by which process. Drop the post-outcome column, retrain, accept the honest score, and only then produce charts. If a leaked chart already reached an audience, issue the correction naming the column and the honest number — the model that quietly ships the leak costs far more than the meeting that kills it.
Make the leakage audit a gate on the interpretation pipeline: no explanation report without a line stating "time-of-knowledge audit done on the top N features". Treat a dominant single feature (over half the credit) as a smell that triggers the same audit. Teach the team's reviewers the one-liner: a beautiful explanation certifies usage, not correctness — interpretation never fixes leakage, it only photographs it.
Q: "Your model hits 0.99 and the SHAP chart looks clean. Ship it?" Strong answer: refuse; a near-perfect score on business tabular data is a leakage alarm, not an achievement; run the time-of-knowledge audit on the top features before any explaining; note that every explanation tool will happily rank a leaked column first because explanations describe usage, not validity; and give the expected honest outcome — drop the leak, retrain, re-explain the humbler model.