Recommender System
The AI that decides what you see next — probably the most economically significant machine learning on earth, and the least discussed.
When not to use it
- When you have few items. If the catalogue is small enough to browse, a recommender is machinery in place of a list.
- When you can't A/B test. Offline metrics don't predict online behaviour. Without a test you're guessing with statistics.
- When the objective hasn't been decided deliberately. You will get exactly what you optimise, at scale, for years. That conversation happens now or it happens in the press.
- On cold start, without a fallback. New users and new items have no signal. Popularity or content-based rules are the honest bridge.
Reach for something else instead
- Search — when users know what they want, let them ask. Recommenders exist for when they don't.
- Editorial curation — humans picking. Better than people admit for small catalogues, and accountable.
- Popularity ranking — the baseline that's embarrassingly hard to beat, and the one people skip measuring against.
- Simple content rules — "more from this creator." Explicable, and often most of the value.
Sources & further reading
- Koren, Bell & Volinsky (2009), Matrix Factorization Techniques for Recommender Systems — the Netflix Prize era, explained clearly by the people who won it.
- Covington, Adams & Sargin (2016), Deep Neural Networks for YouTube Recommendations — the two-stage retrieval-and-ranking shape, from production.
- Chaney, Stewart & Engelhardt (2018), How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility — the feedback loop, modelled.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Trusting offline metrics. The gap between offline gains and online results is the field's most reliable finding.
- Optimising engagement without deciding whether you want what engagement produces.
- Ignoring popularity bias, then discovering the catalogue collapsed to a hundred items.
- No exploration. You can only learn about what you show, and a pure-exploitation system stops learning.
- Treating implicit feedback as preference. A click is not a like, and there's no negative signal at all.
At a glance
FieldApplied AI
Economic weightarguably the largest in ML
Shapetwo-stage retrieval then ranking
Core failurethe feedback loop it created
Offline metricsdon't predict online
DifficultyIntermediate
Flashcards for this concept
Question
—
Answer
—
1 / 4
Often compared with
Recommendation vs. search — search is for when you know what you want; recommendation is for when you don't. Technically they're nearly the same two-stage machinery.