Double Descent
The finding that test error falls, rises, and then falls again as models grow — and that the textbook U-curve was a description of one region, not a law.
Reviewed July 16, 2026Stable
When not to use it
- In the classical regime. With few parameters and plenty of data, the U-curve is an accurate description and you should use it.
- As a reason to skip regularisation. Double descent explains why big models can generalise; it doesn't say regularisation stopped working.
- As an explanation for a specific model's behaviour. It's a phenomenon about a family of models across a capacity axis, not a diagnosis of one training run.
Reach for something else instead
- Held-out validation answers what you actually need — is this model good — without any theory of why.
- The classical bias-variance frame remains correct where it applies, and it applies to most non-deep models.
- Empirical scaling curves for your own task beat any general theory about the shape.
Sources & further reading
- Belkin et al. (2019), Reconciling Modern Machine Learning Practice and the Classical Bias-Variance Trade-off — PNAS; named the curve and showed it beyond neural networks.
- Nakkiran et al. (2019), Deep Double Descent: Where Bigger Models and More Data Hurt — the effect across width, epochs and dataset size in modern networks.
- Zhang et al. (2017), Understanding Deep Learning Requires Rethinking Generalization — the memorisation result that made the classical story untenable in the first place.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Teaching the U-curve as a law. It's the left half of the picture, and modern practice lives on the right.
- Assuming more data is always safe. Adding data shifts the interpolation threshold, and a model that was comfortably past the peak can land on it.
- Reading it as "overfitting isn't real." Overfitting is entirely real; the claim is narrower — that test error is not monotonic in capacity past the interpolation point.
At a glance
FieldMachine Learning
ShapeU, then a peak, then descent again
Peak sits atthe interpolation threshold
Named byBelkin et al., 2019
DifficultyAdvanced