Your recommender only sells what already sells

Nobody decided to concentrate the catalog. It happened one quarter at a time. Two years ago the top 50 products carried 31 percent of revenue; then 38; this quarter, 47. Meanwhile a merchandiser clicking around the site noticed something that the revenue chart was too polite to say directly: the recommendation rail showed roughly the same dozen products on every page. The rail was labeled Discover. (It had not discovered anything since March.)

The inventory ledger tells the rest. Capital sleeping in long-tail stock that gets no impressions, then gets written down. New products taking months to their first organic sale. And the bestsellers, the products that least need help, collecting both the promotional discounts and the free advertising of every recommendation slot, because the system had learned, correctly, that they convert.

A recommender trained on its own recommendations is a mirror, not a merchandiser.

The loop deserves a precise description, because it hides inside good metrics. A recommender learns from clicks and purchases. Clicks and purchases happen on products that get shown. Products get shown because the recommender chose them. So the training data is not a record of demand; it is a record of the model's own past exposure decisions, seasoned with position bias, since whatever sits in slot one gets clicked more for reasons that have nothing to do with preference.

Each training cycle tightens the loop. Popular items gather more evidence of popularity, unexposed items gather nothing, and the model grows statistically certain about a tiny neighborhood of the catalog while knowing nothing about the rest. CTR stays healthy the whole way down, because showing people the thing they were already going to buy is a reliable way to get clicked. The metric rewards the mirror for being a mirror.

The test that exposes it costs one experiment: run your recommender head-to-head against a dumb bestseller list. If the expensive model barely beats the list, that is not because recommendation does not work. It is because your model has spent two years becoming that list.

Specialist teams attack this with three mechanisms, none of which is "buy a better model."

Debias the training data. Inverse-propensity weighting reweights each click by how likely the item was to be shown at all, so an item that converted from rare exposure counts for more than one that converted from permanent residency in slot one. The model starts learning demand instead of echoing exposure.

Buy information with an exploration budget. A small share of recommendation slots goes to under-exposed items with uncertain estimates, allocated by a bandit rather than at random, so the system pays a controlled, measured cost in short term clicks to find out what the rest of the catalog could do. Exploration is not a leak in the funnel; it is the R&D line of the recommender.

Re-rank for the slate, not the slot. Relevance scores pick candidates; a diversity-aware re-ranker composes the final list so it is not four near-identical variants of the same bestseller. Add coverage and new-item time-to-first-sale to the dashboard next to CTR, because what gets a chart gets defended in meetings. Cold-start items, meanwhile, enter with a content-based prior, embeddings from their attributes, text, and images, so a product does not need behavioral history to earn its first impression.

We rebuilt this layer for a retailer whose "also bought" rail had fossilized exactly this way. Long-tail revenue share climbed for five consecutive months, and the metric the finance team actually framed was inventory: stock that moves is capital that stops needing write-downs.

Exploration looks like a tax on this quarter's clicks. Concentration is a tax on every quarter after it.

The honest caveat: exploration has a real short-term cost, and a catalog of two hundred products does not have this problem. This is medicine for catalogs in the thousands, where the unexposed tail is large enough to be hiding real winners.

Ask for the revenue share of your top 50 SKUs, four quarters back. If it climbs every quarter while the catalog grows, your recommender is compounding its own certainty, and the long tail you paid to stock is subsidizing the products that never needed help.

TensorLabs builds recommendation systems with the debiasing, exploration, and re-ranking layers in from the start. If your Discover rail has stopped discovering, write to us with your catalog size and that concentration number. The two together usually say how much the mirror is costing.