Sigmoid Function
The S-shaped curve that squashes any number into the range 0 to 1 — the function that turns a raw score into a probability, and the historical workhorse of neural networks.
Reviewed July 16, 2026Stable
When not to use it
- As the activation in hidden layers of deep networks — it saturates and causes vanishing gradients; use ReLU or its variants.
- For multi-class output — use softmax, its multi-class generalisation.
- When you need outputs that can be negative or unbounded — the 0–1 squash is wrong for those.
Reach for something else instead
- ReLU (and GELU, SwiGLU) for hidden-layer activations — no saturation for positive inputs.
- Softmax for multi-class probability outputs.
- Tanh where a zero-centred (−1 to 1) squash is preferred, though it saturates too.
Sources & further reading
- Rumelhart, Hinton & Williams (1986), Learning representations by back-propagating errors — sigmoid activations in the backpropagation era.
- Glorot & Bengio (2010), Understanding the difficulty of training deep feedforward neural networks — the saturation/vanishing-gradient analysis.
- Goodfellow, Bengio & Courville, Deep Learning — activation functions and their trade-offs.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Using sigmoid activations throughout a deep network and hitting vanishing gradients.
- Applying sigmoid to multi-class outputs instead of softmax (the class probabilities won't sum to 1 correctly).
- Forgetting that saturated outputs (near 0 or 1) carry almost no gradient, stalling learning.
At a glance
FieldDeep Learning
Formulaσ(z)=1/(1+e⁻ᶻ)
Range0 to 1
Modern rolebinary output + gating
Weaknesssaturation → vanishing gradients
DifficultyBeginner