Calibration
Whether a model's confidence means anything — and modern neural networks are worse at it than the ones they replaced.
When not to use it
- (Calibration is a property, not a technique. The question is when to distrust confidence.)*
- Neural network confidence, uncalibrated. Systematically overconfident. Temperature-scale it first; it's ten minutes and free.
- An LLM's verbalised confidence. "I'm quite sure" is generated text, not a probability.
- Confidence-based escalation, without checking. The threshold you set won't fire if the model is overconfident.
- Calibration as a substitute for accuracy. A well-calibrated bad model is honestly bad. That's better than dishonestly bad, and it's still bad.
Reach for something else instead
- Temperature scaling — one parameter, doesn't touch accuracy, fixes most of it.
- Isotonic regression / Platt scaling — for non-neural models.
- Conformal prediction — distribution-free coverage guarantees. Stronger, and gives you sets rather than scores.
- Ensembles — averaging models improves calibration somewhat, for free, if you have them.
Sources & further reading
- Guo et al. (2017), On Calibration of Modern Neural Networks — modern networks are worse calibrated than their less accurate predecessors. Temperature scaling fixes most of it.
- Niculescu-Mizil & Caruana (2005), Predicting Good Probabilities with Supervised Learning — which model families are miscalibrated in which direction, and why.
- Kadavath et al. (2022), Language Models (Mostly) Know What They Know — the more optimistic reading for LLMs, worth weighing against the RLHF findings.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Treating softmax output as probability. It's a normalised score and it's usually overconfident.
- Building confidence-based escalation without checking calibration. Your threshold silently never fires.
- Fitting temperature on training data. It has to be held-out or you've calibrated to the memorised set.
- Assuming better accuracy means better calibration. Guo et al.: they moved in opposite directions.
- Trusting an LLM's stated confidence. It's producing the words a confident person would use.
At a glance
FieldMachine Learning
The claim"90% confident" should mean right 90% of the time
Modern networkssystematically overconfident, worse than their predecessors
The fixtemperature scaling; one parameter, ten minutes, no accuracy cost
Diagnosticreliability diagram
LLMsverbalised confidence is text, not probability
DifficultyIntermediate
Flashcards for this concept
Question
—
Answer
—
1 / 4
Often compared with
Calibration vs. accuracy — one asks whether the answers are right, the other whether the confidence means anything. They're independent, and modern training improved the first while degrading the second.