The model is "great" by the data team's numbers and hated by the people who use it. Dashboards show F1 improving release after release, while the operations or retention team escalates: too many false alarms, wasted vouchers, an overflowing review queue. Both sides bring numbers to the same meeting and talk past each other, because the numbers measure different things.
Any deployment where the modelling team picked the metric alone — F1, accuracy, or AUC chosen "because that is what you optimise" — while the consuming team is judged on an operational cost: vouchers spent, review-hours burned, customers wrongly blocked. Common in churn retention, fraud review, bank collections and lead scoring. The conflict stays hidden during offline development and erupts in the first month of live use, souring trust in the whole ML effort. Linked module: 1.5 (Evaluation & metrics deep dive).
F1 weighs precision and recall equally, which silently assumes a false alarm costs the same as a miss. Here it did not: operations paid per false alarm, so every F1-optimal threshold overspent their budget. Nobody is wrong about their own numbers — the teams optimised different objectives, and the disagreement was designed in on day one, when the metric was chosen without the stakeholders who pay for the errors. Confirm by translating: take the confusion matrix at the shipped threshold, price each cell with operations' own costs, and compare that rupee total across thresholds. If a different gate is clearly cheaper on their costs than the F1-optimal one, the metric — not the model — caused the fight.
Move the meeting before the modelling. Sit with every team that pays for an error and build the cost matrix together: a rupee value on each confusion-matrix cell, written down and signed off. Recompute the operating threshold as the cheapest gate on that matrix — the model itself often needs no change at all. Report in their currency from then on: vouchers spent, churners saved, queue size — with F1 demoted to an internal diagnostic. If two consuming teams carry different costs, ship two thresholds or a top-K queue sized to each team's capacity rather than one compromise gate.
Make "metric + cost matrix agreed with stakeholders" a written gate that blocks modelling from starting, the same way 1.1's framing sheet blocks vague problems. Put the costed threshold sweep in every model review, so the operating point is a business decision with a bill attached. Re-confirm the cost matrix quarterly — voucher prices and review capacity drift. Any metric a stakeholder cannot restate in their own currency is not yet an agreed metric.
Q: "Your model improved F1 by ten points and the business team wants it rolled back. What went wrong?" Strong answer: F1's hidden assumption — equal cost for both error types — stated in one sentence, then the diagnosis that the teams optimised different objectives, then the fix: a stakeholder-agreed rupee cost matrix and a threshold chosen as the cheapest gate on it. Candidates who say "the business does not understand F1" fail; the metric was the thing that did not understand the business.