The top few items keep taking a larger share of sales every month, while the long tail of the catalogue quietly stops selling. The "people also bought" rail shows the same handful of products almost everywhere. Sellers of niche items complain they never surface; catalogue coverage of the recommender — the share of items it ever recommends — keeps shrinking. Individual metrics like click-through can look healthy the whole time.
A recommender retrained on a regular schedule from click and purchase logs — logs that the recommender itself has been shaping since it launched. Whatever the model promotes gets seen more, therefore bought more, therefore looks even stronger in the next training set. No exploration slots, no exposure logging, no diversity or coverage metric on the scorecard. The team's offline gate checks precision@k only. Linked module: 1.12 (recommendation systems intro).
The training data stopped being a measurement of user preference and became a recording of the model's own placements. An item shown in slot one of every rail collects purchases an identical item in obscurity cannot, so raw purchase counts now measure exposure as much as appeal. Retraining on that signal amplifies it: a feedback loop where popular items become more popular because the system made them so. The long tail starves — tail items get no exposure, so they generate no evidence, so they never earn exposure. This is the same mechanism that builds filter bubbles in news feeds; here it eats a catalogue instead of a worldview.
Break the loop from three sides. Reserve exploration slots: a small fixed share of every rail — a slot or two in ten — serves fresh, tail, or random-among-eligible items, giving the starved catalogue a chance to generate evidence. Log exposure, and judge items by conversion given views rather than raw purchase counts, so a tail item seen rarely but converting well can rise. Add diversity to the scorecard: track catalogue coverage and the tail's share of recommendations alongside precision@k, and cap the share of impressions any single item may take.
Whenever training data is produced by the system being trained, treat the loop as the default risk, not an edge case. Keep catalogue coverage and tail share as first-class metrics beside precision@k on the standing dashboard, with a review trigger when coverage falls. Make exploration slots a permanent fixture rather than an experiment to be "optimised away" — short-term click-through will always argue for removing them, and short-term click-through is the voice of the loop.
Q: "Why can training a recommender on its own logs go wrong, and what would you do about it?" Strong answer: name the exposure feedback loop and tell the popularity-inflation story in two sentences, then give the three-part treatment — exploration slots, exposure-corrected signals (conversion per view), and coverage or diversity metrics on the scorecard. Mentioning that filter bubbles are the same loop applied to content feeds shows the idea generalises.