A feature-importance chart is presented to management as "what drives churn" (or default, or fraud). Then it falls apart under questions: two features everyone knows measure the same thing sit mid-table with similar scores, a column nobody believes in (an ID-like field, a random campaign code) ranks above a known driver, and retraining with a different seed reshuffles the order. Leadership stops trusting the model — or worse, acts on the wrong lever.
feature_importances_ from a RandomForestClassifier or gradient-boosting model,
read straight off the fitted model and turned into a bar chart for a review deck. The
table contains correlated near-twins (days since last order and days since last app
open; income and EMI capacity) and high-cardinality numeric columns. Linked module:
1.4 (Trees, RF, boosting, XGBoost); module 1.11 covers the deeper toolkit.
The default attribute is impurity-based importance, and it has three known biases.
(1) Correlated twins split the credit: either column can serve a given split, so each
ends up with roughly half its true share — in the lesson's staged demo, a 0.555 score
became 0.310 + 0.272 the moment a near-twin was added. (2) Columns with many distinct
values offer many candidate cut points, so a pure-noise column can nibble impurity by
chance — the staged lucky_number column scored level with a real feature. (3) The
score is computed from training-side splits, so memorised noise counts as
"importance". Confirm the diagnosis in two steps: run a correlation matrix over the
suspect columns (the twins print 0.99), and run permutation_importance on the TEST
set — noise columns collapse to zero or below while real signal survives.
Rebuild the story before the next meeting. Run permutation_importance on the held-out
test set (n_repeats of 10-20, fixed seed) and rank by mean score drop. Check the
correlation matrix; where two features are near-twins, either drop one, or present
them as one combined driver with their credit added — never as two independent bars.
Rerun across two or three seeds and show only rankings that are stable. If a noise or
ID-like column still ranks high after all this, treat it as a leakage lead (module
1.7), not an insight.
Never brief leadership from feature_importances_ alone — pair it with test-side
permutation importance and a correlation check, as team policy. Keep a deliberate
random "canary" column in exploratory runs: the day it ranks above a real feature,
your chart is not ready. Present importance as "what the model leaned on", never as
proof of cause — the association-versus-cause caution from 1.3 applies to trees
unchanged.
Q: "Your random forest says feature X is the most important driver. How much do you trust that?" Strong answer: name impurity importance's biases (credit-splitting across correlated features, inflation of many-valued columns, training-side computation), give the cross-check (permutation importance on the test set plus a correlation matrix), and close with the association-not-cause caveat. Candidates who mention a canary noise column, or that permutation importance also struggles with correlated twins, show real practice rather than textbook recall.