Confusion Matrix
A table of what got classified as what — the least sophisticated tool in evaluation, and the one that tells you the most.
When not to use it
- For regression. It's a classification tool. Predicting a number needs error distributions.
- Unnormalised, on imbalanced data. The majority class swamps every cell and the rare class vanishes.
- With hundreds of classes. An n×n table stops being readable. Sort by error mass and look at the top confusions.
- As a headline number. It isn't one. That's a feature, and it's why it doesn't get reported.
Reach for something else instead
- Cost-weighted error — the matrix times what each mistake actually costs. The honest version.
- Per-class precision and recall — the row- and column-normalised views, as numbers.
- Top-k confusions — for many classes, list the biggest off-diagonal cells.
Sources & further reading
- Fawcett (2006), An Introduction to ROC Analysis — the clearest treatment of what the four cells are and what follows from them.
- Powers (2011), Evaluation: From Precision, Recall and F-Measure to ROC, Informedness, Markedness & Correlation — a careful account of what each summary metric throws away.
- Provost & Fawcett (2013), Data Science for Business — cost-weighted evaluation, and why the cost matrix is the conversation people avoid.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Not looking at it. It's a table; reading it feels unsophisticated, and it finds more than any metric.
- Leaving it unnormalised on imbalanced data, so the rare class is invisible.
- Reporting accuracy and skipping the matrix, then being surprised by a class the model never predicts.
- Never building a cost matrix, so every error is implicitly worth the same. It isn't.
At a glance
FieldMachine Learning
What it isactual class vs. predicted class, as a table
What it containsevery metric you've heard of, before compression
Alwaysnormalise by row or column
The honest extensionmultiply by a cost matrix
Why it's unreportedit isn't a number you can rank
DifficultyBeginner
Flashcards for this concept
Question
—
Answer
—
1 / 4
Often compared with
Confusion matrix vs. accuracy — one shows you every mistake and which; the other compresses all of it into one division. Only one of them tells you what to fix.