Confusion Matrix
A table of what got classified as what — the least sophisticated tool in evaluation, and the one that tells you the most.
When not to use it
- For regression. It's a classification tool. Predicting a number needs error distributions.
- Unnormalised, on imbalanced data. The majority class swamps every cell and the rare class vanishes.
- With hundreds of classes. An n×n table stops being readable. Sort by error mass and look at the top confusions.
- As a headline number. It isn't one. That's a feature, and it's why it doesn't get reported.
Reach for something else instead
- Cost-weighted error — the matrix times what each mistake actually costs. The honest version.
- Per-class precision and recall — the row- and column-normalised views, as numbers.
- Top-k confusions — for many classes, list the biggest off-diagonal cells.
A confusion matrix lays out every kind of right and wrong.
The grid crosses what the model predicted against what was true. The diagonal is correct (true positives and true negatives); the off-diagonal is the two ways to be wrong — false positives (crying wolf) and false negatives (missing the wolf). Almost every classification metric — precision, recall, F1 — is just a ratio computed from these four boxes.
Worked example
Say a spam filter is tested on 100 emails: 40 are actually spam, 60 are not. The filter's results:
Missed spam, called it fine (FN): 5
Good mail wrongly flagged (FP): 8
Good mail correctly passed (TN): 52
Now every metric falls out of these four numbers:
precision = 35 / (35+8) = 35/43 = 0.81
recall = TP / (TP+FN)
recall = 35 / (35+5) = 35/40 = 0.88
accuracy = (TP+TN) / total
accuracy = (35+52) / 100 = 0.87
Read it plainly: when this filter says "spam," it's right 81% of the time (precision); of all real spam, it catches 88% (recall). The single accuracy number, 87%, hides that trade — which is exactly why the matrix matters more than accuracy alone.
Further reading
- Fawcett (2006), An Introduction to ROC Analysis — the clearest treatment of what the four cells are and what follows from them.
- Powers (2011), Evaluation: From Precision, Recall and F-Measure to ROC, Informedness, Markedness & Correlation — a careful account of what each summary metric throws away.
- Provost & Fawcett (2013), Data Science for Business — cost-weighted evaluation, and why the cost matrix is the conversation people avoid.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Not looking at it. It's a table; reading it feels unsophisticated, and it finds more than any metric.
- Leaving it unnormalised on imbalanced data, so the rare class is invisible.
- Reporting accuracy and skipping the matrix, then being surprised by a class the model never predicts.
- Never building a cost matrix, so every error is implicitly worth the same. It isn't.
At a glance
Often compared with
Where this sits
2 concepts come first. Understanding it opens up 6 more.
Computed from the prerequisite graph, not assigned. How this works