Model Monitoring
Watching a deployed model for the failures that don't raise errors — and the thing you most need to watch is the thing you can't see.
When not to use it
- (Monitoring choices that mislead.)*
- Input drift as a proxy for accuracy. It fires when nothing matters and misses what does.
- Statistical significance as an alert threshold. At scale everything is significant. Use effect size.
- Aggregate metrics only. Your model can be fine overall and broken for one segment entirely.
- Standard APM alone. It was built for systems that crash. This one returns 200 and lies.
Reach for something else instead
- Prediction drift — the best signal per unit of effort, and most teams lack it.
- Labelled sampling — slow, expensive, the only thing that measures truth.
- Conformal prediction — coverage guarantees, no labels needed. Underused.
- Shadow deployment — compare against the incumbent on real traffic before switching.
Sources & further reading
- Breck et al. (2017), The ML Test Score — the monitoring axis; most teams score near zero.
- Rabanser, Günnemann & Lipton (2019), Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift — test the model's representation, not raw features.
- Angelopoulos & Bates (2023), Conformal Prediction: A Gentle Introduction — coverage guarantees without labels or distributional assumptions.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Expecting failures to be loud. It returns 200 with a confident wrong answer.
- Monitoring inputs and calling it monitoring. That's a hint, not a measurement.
- Alerting on p-values at scale. You'll drown, then you'll mute it.
- Not noticing your alerts are caused by your responses to alerts. Retraining shifts the distribution that triggers the monitor.
At a glance
FieldApplied AI
The problemmodels fail silently and return 200
The central difficultyaccuracy needs labels; labels arrive late or never
The best cheap signalprediction drift, at the output
The rulealert on effect size, never significance
The unsolved caseLLMs; all proxies
DifficultyIntermediate
Flashcards for this concept
Question
—
Answer
—
1 / 4
Often compared with
Input drift vs. prediction drift — one fires when features move whether or not it matters; the other only moves when something reached the output. Watch the second.