Why does deep learning work? The generalization mystery

Here is a fact that should be more unsettling than it usually sounds: the most consequential technology of the decade rests on a foundation that its own creators do not fully understand. Deep learning works, spectacularly, and yet there is no complete theory explaining why it works as well as it does. By the standards of the mathematics that was supposed to govern it, the enormous neural networks behind modern AI should not work at all. They are so oversized that classical learning theory predicts they should memorize their training data and then fail on anything new. Instead they generalize beautifully to examples they have never seen, and the gap between what theory predicts and what these systems actually do is one of the deepest open problems in the field.

This is the generalization mystery, and it deserves to be understood by anyone who wants to know how solid the ground beneath modern AI really is. This piece explains the puzzle precisely: why deep networks, by all classical reasoning, ought to overfit and fail; the strange empirical phenomena, double descent, grokking, and benign overfitting, that overturned the textbook picture; the leading explanation for why generalization happens anyway; and the honest state of a question that remains unresolved. It is a story about a technology that works better than our best theory can account for, which is both a triumph and a humbling admission.

The setup: models big enough to memorize anything

To feel the mystery, you first have to appreciate how oversized modern neural networks are. They are overparameterized, meaning they have far more internal parameters, the adjustable numbers that store what they learn, than they have training examples to learn from. A network may have billions of parameters and be trained on a smaller number of data points. In classical terms, this is an alarming amount of capacity: more than enough to simply memorize the entire training set rather than learn any general pattern from it. And in fact these networks do drive their error on the training data down to essentially zero, fitting every example perfectly.

The most striking demonstration of just how much raw memorization capacity they have came from an experiment that has haunted learning theory ever since. Researchers took standard image networks and trained them on data whose labels had been randomly shuffled, so that there was no real pattern connecting images to labels at all, only noise. The networks fit this random data perfectly too, achieving zero training error on pure nonsense. This proved something important: these models have enough capacity to memorize arbitrary data, with no underlying structure whatsoever. Whatever they are doing on real data, it is not that they lack the ability to memorize. They could memorize; the question is why, on real problems, they seem to do something better.

The puzzle: they should overfit, but they do not

Now the mystery snaps into focus. A model with enough capacity to memorize random noise should, by every classical expectation, overfit real data as well: it should fit the training examples exactly, including whatever noise and quirks they contain, and then perform poorly on new examples because it memorized specifics rather than learning generalizable structure. This is the central warning of classical machine learning, the reason practitioners were long taught to limit model size and avoid fitting the training data too closely.

Yet overparameterized deep networks do the opposite. They fit the training data perfectly and generalize well to unseen data, often better as they get larger. The puzzle has two parts, and both matter. First, why do these enormous models not overfit, when their capacity says they should? Second, given that there are astronomically many different ways to perfectly fit the training data, most of which would amount to useless memorization that fails on new examples, why does the training process reliably find one of the rare solutions that generalizes? Something is steering these networks, out of all the ways they could fit the data, toward the ways that work on data they have never seen. Identifying that something is the heart of the problem.

Why this broke the textbook

The reason this was so disruptive is that it contradicts the framework that governed machine learning for decades: the bias-variance tradeoff. The classical picture says model performance follows a U-shaped curve as you increase complexity. Too little capacity and the model underfits, unable to capture the pattern, doing badly on both training and test data. Too much capacity and it overfits, memorizing the training data and doing badly on test data. Somewhere in between is a sweet spot that minimizes error on new data. The clear practical lesson was to tune model size to that sweet spot and to be suspicious of any model that fit its training data perfectly, because perfect fit was taken as a sign of overfitting.

Formal versions of this reasoning, using classical measures of model complexity, all pointed the same way: capacity far exceeding the training data should destroy generalization. Deep learning violates this at every turn. Modern networks sit far into the overparameterized regime the theory warned against, fit their training data completely, and generalize better than the smaller models the U-curve would recommend. The classical complexity measures, including simply counting parameters, turned out to be poor predictors of how well a deep network generalizes. The theory was not merely incomplete; on its own terms it made the wrong prediction, which is why the mystery forced a rethinking rather than a footnote.

Double descent: the curve that replaced the U

The first major crack in the old picture came from a phenomenon called double descent. Researchers found that the classical U-shaped curve is only the left half of a larger shape. As you increase model size, test error does first decrease and then increase, exactly as the U predicts, reaching a peak at what is called the interpolation threshold, the point where the model becomes just big enough to fit the training data exactly. But if you keep making the model larger, past that threshold, test error does something the classical theory forbids: it descends again. This is the second descent that gives the phenomenon its name.

The implication is remarkable. The models that generalize best are not the medium-sized ones at the classical sweet spot but the very large ones far beyond the interpolation threshold, in the regime the textbook said to avoid. Double descent reconciles the modern experience that bigger models keep getting better, which the scaling laws describe, with the classical U that only ever saw the first half of the curve. The same behavior appears not only as model size grows but also as training continues over time, and it has been observed well beyond neural networks, in simpler models too, suggesting it is a general feature of overparameterized learning rather than a quirk of deep networks. Double descent did not by itself explain the mystery, but it decisively showed that the old map was wrong.

Grokking: understanding that arrives late

If double descent bent the classical picture, a phenomenon called grokking broke it in a different and stranger way, this time in the dimension of training time. In grokking, a network is trained on a task and quickly memorizes the training data, driving training accuracy to perfect while its accuracy on held-out test data stays near chance, exactly what memorization without understanding looks like. In the classical view, that is the end of the story: the model has overfit and further training should not help. But if training is allowed to continue, long after the training loss has flattened and it appears nothing more is happening, something extraordinary can occur. Thousands of steps later, test accuracy suddenly jumps from near-random to nearly perfect. The model abruptly transitions from memorizing to truly generalizing, as if it has finally understood the underlying rule.

Grokking is important beyond its strangeness because of what it reveals. It shows that memorization and generalization are distinct processes that can happen at very different times, that a network can be quietly reorganizing its internal representations even when its loss looks completely stagnant, and that the point at which a model appears to have finished learning may not be the point at which it has learned to generalize. It is a vivid demonstration that we cannot read a model's true understanding off its training curve, and it has become a favorite testbed for researchers trying to catch generalization in the act of forming.

Benign overfitting and the leading explanation

Alongside these sits a third classical-defying finding, benign overfitting: the observation that an overparameterized model can fit its training data perfectly, including any noise in it, without the noise harming its performance on new data the way classical theory insists it must. Overfitting, in this regime, can be harmless. Taken together, double descent, grokking, and benign overfitting make clear that the classical framework is missing something fundamental about how these models behave.

So what is the missing ingredient? The leading answer is the implicit bias of the training process, sometimes called implicit regularization. The idea is this: among the astronomically many solutions that all fit the training data perfectly, gradient descent, the algorithm used to train networks, does not choose randomly. It is quietly biased toward particular kinds of solutions, and the ones it favors tend to be simple, in a sense that corresponds to smooth, low-complexity functions rather than jagged memorizing ones. No one adds an explicit rule forcing this; it falls out of how gradient descent moves through the space of possible networks. So although a network has enormous capacity to represent complicated functions, the training process implicitly constrains its effective complexity, steering it toward simple solutions that happen to generalize. This is why the interaction matters: generalization is not a property of the architecture alone, nor the data alone, but of the architecture, the data, and gradient descent working together.

A useful way to see this is through simplicity bias: networks trained by gradient descent tend to learn simple, broadly applicable patterns before resorting to memorizing specifics. On real data, which contains genuine learnable structure, the simple solution the optimizer prefers is one that generalizes. On random data, which has no structure to find, the network is forced all the way to memorization, which is exactly what the random-label experiment showed. The same mechanism explains both results. Related observations, such as the tendency of the optimizer to settle in flat rather than sharp regions of the error landscape, which is associated with better generalization, fill in the picture. The effective complexity of a deep network is far smaller than its parameter count suggests, because the way it is trained keeps it simple.

The honest state of the question

It would be a misrepresentation to tell this story as solved. Implicit bias is the leading explanation, and it is supported by a great deal of evidence, but it is not yet a complete, predictive theory from which one could derive, in advance, how well a given network will generalize. The field can characterize the phenomena, double descent, grokking, benign overfitting, far better than it can explain them from first principles, and there is active disagreement about how deep the mystery really is. Some researchers argue that these behaviors are less anomalous than they first appeared and can be captured by refined versions of long-standing generalization frameworks, that deep learning is not so different after all. Others hold that it demands entirely new theoretical foundations. This debate is unsettled, and being honest that it is unsettled is more useful than pretending otherwise.

What can be said cleanly is this: deep learning works better than our best theory can currently explain, and closing that gap remains one of the most important open problems in machine learning. This connects to the broader difficulty of understanding these systems from the inside, the goal of interpretability research; in both cases we have built something powerful and are now, after the fact, trying to work out how it does what it does. The generalization mystery is the theoretical face of that same humbling situation.

Why it matters

This is not merely a puzzle for theorists to enjoy. A real understanding of why deep networks generalize would have practical force. It would help predict when a model will fail to generalize rather than discovering it after deployment, guide the design of models that reach a given capability with far less data and compute, and tell us how much to trust a system in situations unlike its training data. It also bears on safety: guarantees about a system's behavior are far harder to make when we cannot explain why it behaves well in the first place, and a technology given real responsibility ideally rests on more than the observation that it has worked so far. There is, finally, a plain intellectual humility in the situation worth sitting with. The defining technology of the era runs on principles we are still reverse-engineering, which is a reminder that capability and understanding are not the same thing, and that we have raced ahead on the first while the second is still catching up.

The short version

Deep learning poses a real puzzle for theory: its neural networks are so overparameterized that they can memorize even randomly labelled data, which by classical reasoning means they should overfit real data and fail to generalize. Instead they fit their training data perfectly and generalize well to unseen data, often improving as they grow larger. This violates the classical bias-variance tradeoff and its U-shaped curve, and it was made undeniable by three phenomena: double descent, in which test error falls again as models grow past the point of perfectly fitting the data; grokking, in which generalization appears suddenly long after a model has memorized and its loss has flattened; and benign overfitting, in which fitting noise does not harm performance. The leading explanation is the implicit bias of gradient descent, which, among the many solutions that fit the training data, favors simple ones that generalize, so a network's effective complexity is far smaller than its parameter count. But there is no complete theory, and the question remains open.

The idea to hold onto is that deep learning works better than our best theory says it should, because the way these networks are trained quietly biases them toward simple, generalizing solutions rather than the memorization their raw capacity would allow, and understanding exactly why remains one of the deepest unsolved problems in AI. We built the most important technology of the decade first and are still working out why it works, which is worth remembering whenever it is described as though it were fully understood.

Common questions

Why do neural networks generalize? The leading explanation is the implicit bias of the training process. A neural network has enough capacity to fit its training data in astronomically many ways, most of which would be useless memorization, but gradient descent, the algorithm used to train it, does not choose among them randomly. It is quietly biased toward simple, smooth solutions rather than jagged memorizing ones, and on real data those simple solutions tend to generalize. So although the network's raw capacity is enormous, the way it is trained constrains its effective complexity, steering it toward solutions that work on unseen data. This is not yet a complete theory, but it is the best current account.

What is the generalization mystery in deep learning? It is the puzzle of why deep neural networks generalize well when classical theory says they should not. Modern networks are overparameterized, with far more parameters than training examples, enough capacity to memorize their training data entirely, which they demonstrably can, since they can fit even randomly labelled data. Classical learning theory predicts that such models should overfit real data and fail on new examples. Instead they generalize well, often better as they grow larger. The mystery is both why they avoid overfitting and how training reliably finds, among all the ways to fit the data, one of the rare solutions that generalizes.

What is double descent? Double descent is a phenomenon that overturned the classical picture of model complexity. The traditional bias-variance view says test error follows a U-shaped curve as a model grows: decreasing, then increasing due to overfitting, with a sweet spot in between. Double descent shows this is only half the curve. Test error peaks at the interpolation threshold, where the model becomes just large enough to fit the training data exactly, but as the model grows beyond that point, test error descends a second time. This means the largest, most overparameterized models can generalize best, contradicting the classical advice to avoid perfectly fitting the training data.

What is grokking in machine learning? Grokking is a phenomenon in which a network generalizes suddenly, long after it has memorized its training data. Early in training, the model fits the training data perfectly while its accuracy on new data stays near chance, the signature of memorization without understanding. If training continues well past the point where the training loss has flattened and nothing seems to be happening, test accuracy can abruptly jump from near-random to nearly perfect, thousands of steps later. The model appears to shift from memorizing to actually understanding the underlying rule. Grokking shows that memorization and generalization are distinct and that a network can keep reorganizing internally even when its loss looks stagnant.

What is overparameterization? Overparameterization means a model has far more parameters, the adjustable numbers that store what it learns, than it has training examples. Modern deep networks are heavily overparameterized, often with billions of parameters, giving them enough capacity to memorize their entire training set and fit it to zero error. In classical machine learning this was considered dangerous, because such capacity was expected to cause overfitting. A central surprise of deep learning is that overparameterized networks generalize well despite this, and that generalization can even improve as models become more overparameterized, which is the opposite of what the classical theory predicted.

Why doesn't deep learning overfit like the textbook predicts? Because having the capacity to overfit is not the same as being driven to overfit, and the training process matters as much as the model's size. Although an overparameterized network could memorize its training data uselessly, gradient descent is implicitly biased toward simple solutions among all those that fit the data, and simple solutions tend to generalize on real, structured data. The network's effective complexity is therefore much lower than its parameter count implies. Phenomena like double descent and benign overfitting confirm that, in the overparameterized regime, perfectly fitting the training data does not carry the penalty classical theory assumed.

Do we actually understand why deep learning works? Not completely. We have a leading explanation, the implicit bias of gradient descent toward simple, generalizing solutions, supported by strong evidence, and we can describe the surprising phenomena, double descent, grokking, and benign overfitting, in detail. But there is no complete, predictive theory that lets us derive in advance how well a given network will generalize, and researchers disagree about how fundamental the mystery is, with some arguing refined classical frameworks suffice and others that new theory is needed. The honest summary is that deep learning works better than our best theory can currently explain, and closing that gap is an open problem.