Precision and Recall
The two ways to be right and the two ways to be wrong — and the trade-off that accuracy hides from you.
When not to use it
- On regression problems. These are classification metrics. Predicting a number needs error metrics, not precision.
- When classes are balanced and errors cost the same. Then accuracy is fine and simpler.
- F1 specifically, when your costs are asymmetric. It assumes precision and recall matter equally, which is almost never true.
- Aggregate-only, on multi-class problems. The average hides the class that's failing.
Reach for something else instead
- Accuracy — fine when balanced and symmetric. Dangerous otherwise.
- PR-AUC — better than ROC-AUC on imbalanced data.
- Cost-weighted metrics — put actual money on each error type. The most honest version.
- Precision@k / Recall@k — for ranked output like search, where only the top few matter.
Sources & further reading
- van Rijsbergen (1979), Information Retrieval — the classical treatment; where the F-measure comes from.
- Saito & Rehmsmeier (2015), The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets — the case for PR curves, made carefully.
- Chouldechova (2017), Fair Prediction with Disparate Impact — the impossibility result: you cannot equalise all the error rates at once.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Reporting accuracy on imbalanced data. A 99% score can mean "predicts no every time."
- Optimising F1 by default, then discovering the system was tuned for a trade-off nobody wanted.
- Reporting ROC-AUC on a heavily imbalanced problem, which flatters the model.
- Not choosing a threshold deliberately. Leaving it at 0.5 is a decision, just an unconsidered one.
- Aggregate-only reporting, which hides the rare-class failure that motivated the project.
At a glance
FieldMachine Learning
Precisionof what you flagged, how much was right
Recallof what was there, how much you caught
The trade-offyou cannot maximise both
The knobthe threshold
F1their harmonic mean, and a poor default
DifficultyBeginner
Flashcards for this concept
Question
—
Answer
—
1 / 4
Often compared with
Precision vs. recall — not being wrong vs. not missing anything. Every real system chooses, and the choice is about cost, not mathematics.