Softmax
Turning a list of arbitrary numbers into a probability distribution — the operation at the end of every classifier and inside every attention head.
Reviewed July 16, 2026Stable
When not to use it
- For multi-label problems where options are not mutually exclusive. Use independent sigmoids.
- As a confidence estimate without calibration. High softmax probability and correctness are only loosely related.
Reach for something else instead
- Sigmoid per class for multi-label classification.
- Sparsemax where genuinely zero probabilities are wanted.
- Linear attention kernels where the goal is avoiding the quadratic cost.
Sources & further reading
- Bridle (1990), Probabilistic Interpretation of Feedforward Classification Network Outputs — the original framing.
- Vaswani et al. (2017), Attention Is All You Need — scaled dot-product attention and why the scaling factor exists.
- Guo et al. (2017), On Calibration of Modern Neural Networks — why softmax outputs are not trustworthy confidence.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Reading the output as calibrated confidence. Modern networks are typically overconfident.
- Applying softmax twice, which happens when a loss function already includes it.
- Forgetting the max-subtraction and hitting overflow on large logits.
At a glance
FieldDeep Learning
Convertsscores to a probability distribution
Appears inclassifier outputs and attention
Pairs withcross-entropy
Controlled bytemperature
DifficultyHands-on
Where this sits
5 concepts come first. Understanding it opens up 70 more.
5Levelsteps in
5Needs firstconcepts
70Opens up27% of 264
8Areascrosses fields
Learn these firstActivation Function
Computed from the prerequisite graph, not assigned. How this works