Double Descent
The finding that test error falls, rises, and then falls again as models grow — and that the textbook U-curve was a description of one region, not a law.
Reviewed July 14, 2026Stable
When not to use it
- In the classical regime. With few parameters and plenty of data, the U-curve is an accurate description and you should use it.
- As a reason to skip regularisation. Double descent explains why big models can generalise; it doesn't say regularisation stopped working.
- As an explanation for a specific model's behaviour. It's a phenomenon about a family of models across a capacity axis, not a diagnosis of one training run.
Reach for something else instead
- Held-out validation answers what you actually need — is this model good — without any theory of why.
- The classical bias-variance frame remains correct where it applies, and it applies to most non-deep models.
- Empirical scaling curves for your own task beat any general theory about the shape.
Read more on the blog
- Why neural networks generalize when theory says they can'tA network with more parameters than training examples can memorize random labels perfectly. The same network, on real data, generalizes. Classical learning theory has no account of why, and the reason it fails is more precise than "it was wrong."
- Why does deep learning work? The generalization mysteryThe most important technology of the decade rests on a foundation we do not fully understand. By the textbook, deep networks are so oversized they should memorize their training data and fail on everything else. Instead they generalize beautifully, and nobody can fully explain why. This is the generalization mystery, and it is one of the deepest open problems in AI.
Further reading
- Belkin et al. (2019), Reconciling modern machine-learning practice and the classical bias–variance trade-off — PNAS; named the curve and showed it beyond neural networks.
- Nakkiran et al. (2019), Deep Double Descent: Where Bigger Models and More Data Hurt — the effect across width, epochs and dataset size in modern networks.
- Zhang et al. (2017), Understanding Deep Learning Requires Rethinking Generalization — the memorisation result that made the classical story untenable in the first place.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Teaching the U-curve as a law. It's the left half of the picture, and modern practice lives on the right.
- Assuming more data is always safe. Adding data shifts the interpolation threshold, and a model that was comfortably past the peak can land on it.
- Reading it as "overfitting isn't real." Overfitting is entirely real; the claim is narrower — that test error is not monotonic in capacity past the interpolation point.
At a glance
FieldMachine Learning
ShapeU, then a peak, then descent again
Peak sits atthe interpolation threshold
Named byBelkin et al., 2019
DifficultyAdvanced
Where this sits
A destination. 6 concepts lead here, and nothing in the corpus depends on it.
6Levelsteps in
6Needs firstconcepts
0Opens upnothing further
1Areastays here
Learn these firstBias-Variance Tradeoff
Computed from the prerequisite graph, not assigned. How this works