Anomaly Detection
Finding the unusual thing — where the base rate makes precision nearly impossible and almost every deployment drowns in false alarms.
When not to use it
- When you can label examples. Then it's imbalanced classification, which is a better-understood problem with better tools.
- When the review capacity doesn't exist. A system producing more alerts than anyone can read is worse than none — it teaches people to ignore alarms.
- When "unusual" isn't what you want. You want important. Those overlap partially and the difference is where these projects fail.
- On dirty training data, with reconstruction methods. If anomalies are in the training set, the model learns them as normal.
Reach for something else instead
- Rules — if you know what fraud looks like, write the rule. Faster, explicable, auditable.
- Imbalanced classification — whenever you have labels, this is the better frame.
- Control charts — decades old, readable by anyone, hard to beat on a well-behaved metric.
- Forecast residuals — the natural frame for time series.
Sources & further reading
- Chandola, Banerjee & Kumar (2009), Anomaly Detection: A Survey — the reference taxonomy; point, contextual, collective.
- Liu, Ting & Zhou (2008), Isolation Forest — the strong default, and unusually simple.
- Wu & Keogh (2021), Current Time Series Anomaly Detection Benchmarks are Flawed and are Creating the Illusion of Progress — the critique that the field's evaluation is broken.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Ignoring the base rate, then being surprised by the false alarm volume. It's arithmetic.
- Deploying without an alert budget. The number of alerts a human can review is the actual design constraint.
- Assuming training data is clean. Anomalies in it become invisible by construction.
- Treating "statistically unusual" as "operationally important." Only a person can close that gap.
- Believing benchmark results. The field's evaluation is contested and much reported progress may not transfer.
At a glance
FieldApplied AI
The hard partthe base rate, not the model
Typespoint, contextual, collective
Design constrainthow many alerts a human can review
Without labelsyou cannot validate it
Strong defaultIsolation Forest
DifficultyIntermediate
Flashcards for this concept
Question
—
Answer
—
1 / 4
Often compared with
Anomaly detection vs. classification — if you can label examples, use classification. Anomaly detection is what you do when you can't, and it's much harder to trust.