What is synthetic data? Training AI on AI-made data

There is a cliffhanger built into the story of how AI scaled. The scaling laws that drove the last several years demand ever more data, but the supply of high-quality human-written text is finite, and the field is approaching the point of exhausting it, a limit known as the data wall. So where does the next wave of training data come from when the world runs out of the human kind? One of the most important answers is data that was never collected from the world at all, but generated: synthetic data, produced by AI models, simulations, or rules, and then used to train other models. It has quietly moved from a niche trick to a central part of how frontier systems are built, and it arrives with a famous and frightening warning called model collapse.

The whole story of synthetic data lives in the tension between its promise and that warning, and the resolution of that tension is one of the more important ideas in modern AI. This guide explains what synthetic data is, why labs increasingly rely on it, how it is generated, the model collapse phenomenon that seems to doom the whole idea, and the key distinction, verification and curation, that separates synthetic data that destroys a model from synthetic data that dramatically improves it. It answers a question the scaling-laws piece left open: what happens when we run out of data, and can machines make their own?

What synthetic data is

Synthetic data is data that is generated artificially rather than collected from the real world. Instead of gathering text written by people, images captured by cameras, or records logged by real systems, you produce the data with a generator: an AI model that writes example text, a simulator that renders scenes, or a set of rules that fabricates plausible records. That generated data is then used to train a model, standing in for or supplementing data collected from reality. The idea is not new in principle, engineers have long augmented datasets with transformed copies of real examples, but its scope has expanded dramatically, to the point where models now generate large bodies of fresh training material for other models, and increasingly for themselves.

Why labs increasingly rely on it

Several forces push toward synthetic data, and the data wall is only the most dramatic. The first is simple supply: when high-quality human data becomes scarce or has been largely used, generating more is one of the few ways to keep feeding data-hungry training. The second is coverage: real data is unevenly distributed, thin on exactly the rare and difficult cases that matter most, and synthetic generation lets you deliberately produce examples of situations that are uncommon in the wild, such as unusual edge cases for a self-driving system. The third is privacy: synthetic data can be generated to mimic the statistical shape of sensitive real data, like medical or financial records, without exposing any real person's information, which matters as privacy law tightens. And the fourth is control and cost: generating data can be cheaper and faster than collecting and labelling it, and it can be targeted precisely at a model's weaknesses. Together these have made synthetic data less an option than a necessity at the frontier.

How synthetic data is generated

There are several ways to produce it, suited to different kinds of data. The most common today is to use a capable language model as the generator: prompt a strong LLM to produce example texts, instructions paired with responses, or step-by-step reasoning traces, and collect its outputs as training data. Closely related is knowledge distillation, where a strong teacher model generates outputs that train a smaller student, which is synthetic-data generation by another name. Simulators are used where the world can be modelled, physics and graphics engines generate labelled scenes for robotics and autonomous vehicles, complete with perfect ground truth that would be costly to annotate by hand. Rule-based generators fabricate structured records for tabular and database-style data. And generative image models produce synthetic pictures. Across all of these, the recent shift is that language-model generation has become dominant for text and reasoning data, because a strong model can produce rich, contextually appropriate examples with little effort, which is exactly what makes the collapse question so pressing.

Where it is used in training

Synthetic data now appears throughout the training pipeline. It has long been used for instruction tuning, where a model is taught to follow instructions using generated instruction-and-response pairs, and for alignment, where generated comparisons help build the preference data behind reinforcement learning from human feedback. Its fastest-growing and most consequential use is in reasoning: models generate large numbers of worked solutions to problems, and the good ones become training data that teaches stronger reasoning, which is a major reason reasoning models have advanced so quickly. Synthetic data has even shown striking results in pretraining itself. A well-known line of work produced small but capable models by training them on synthetic "textbook-quality" material, carefully generated to be clear, diverse, and educationally structured, and found that this curated synthetic data delivered outsized gains at a fraction of the usual data volume. That result points directly at the lesson the collapse debate teaches: what matters is not how much synthetic data you generate but how good it is.

The warning: model collapse

Now the danger. If a model generates data and then trains on it, and the next model trains on that, a feedback loop forms, and it can go badly wrong. In a widely cited study, researchers formally described and named model collapse: the finding that indiscriminate training on model-generated content causes progressive, and eventually irreversible, degradation. The mechanism has two stages. In early collapse, small errors in the generated data accumulate and the model drifts away from the true distribution of real data. In late collapse, the tails of the distribution, the rare and unusual cases, progressively vanish, because a model is unlikely to generate the rare events it seldom saw, so each generation of synthetic data is a little less diverse than the last, and training on it makes the next model narrower still. The loop amplifies the model's own mistakes and erodes its variety, and the effect is not unique to language models but appears across generative model families, making it a fundamental risk of naive recursive training. Taken at face value, this seems to sink the whole idea: if training on generated data degrades models, synthetic data cannot be the answer to the data wall.

The resolution: the operative word is "indiscriminate"

Here is the distinction that resolves the tension and is the single most important thing to understand about synthetic data. Model collapse is caused by indiscriminate training on unverified model output fed recursively back into training with no external signal. That is a specific, avoidable procedure, not an inherent property of all synthetic data. When practitioners generate synthetic data with discipline, the collapse largely disappears, and the data becomes powerfully useful. Three practices make the difference.

The first is curation over volume. The lesson of the textbook-quality results is that a smaller amount of carefully generated, diverse, high-quality synthetic data beats a larger amount of careless generation, because volume without curation only amplifies the failure modes while quality per example adds real signal. The second is verification and filtering: rather than training on everything a model generates, you filter its outputs through an evaluator that checks correctness, faithfulness, and diversity, keeping only what passes and discarding the errors before they can be learned. The third is grounding in an external signal: mixing generated data with real data, or validating generated data against ground truth, tools, or human judgment, keeps the process anchored to reality. There is even a theoretical backbone to this. Analyses have shown that as long as a training loop preserves some fraction of real data, or injects a reliable external correctness signal, it stays bounded away from collapse and can converge toward a better model, whereas a loop of pure, unverified synthetic data drifts without limit. The difference between collapse and improvement is not human data versus synthetic data; it is whether the synthetic data carries a genuine signal or merely recycles the model's own errors.

Why it works when done right

This raises a natural puzzle: if a model generates its own training data, where does new information come from? A model cannot bootstrap knowledge out of nothing, so how can training on its own outputs make it better? The answer is that the new information comes from the verification step, not the generation step. For many important tasks, it is far easier to check whether an answer is correct than to produce a correct answer in the first place, a gap between generating and verifying that is especially wide in domains like mathematics, code, and logic, where correctness can be checked mechanically. A model may produce a correct solution only occasionally, but if you generate many attempts and keep only the ones that pass a verifier, you distill a clean, correct signal that the model could produce but not reliably, and training on that curated signal makes it reliable. The information was latent in the model's occasional successes; verification is what extracts it. This is why synthetic data works best exactly where correctness is checkable, and why it is riskier in open-ended domains where "correct" is fuzzy and no clean verifier exists. The verifier, the tool, the ground truth, or the retained human data is the true source of the improvement, and the generated data is the vehicle that carries it into training.

The honest state of things

Synthetic data has become non-negotiable at the frontier. The rapid progress of reasoning models in particular rests heavily on generating and verifying vast amounts of synthetic problem-solving data, and curated synthetic data is now woven through instruction tuning, alignment, and even pretraining. But it is not a free lunch, and the collapse research is a real warning, not an outdated scare. Used naively, recycled unverified model output degrades models irreversibly. Used with discipline, curation for quality and diversity, verification to filter errors, and grounding in an external signal, it is one of the most powerful tools in modern training. The entire practical art lies in that distinction. Synthetic data does not conjure knowledge from nothing; it reorganizes and amplifies signal that some external check can certify, which means the future of training past the data wall depends less on generating more data than on building better verifiers for it.

The short version

Synthetic data is data generated artificially, by AI models, simulators, or rules, rather than collected from the real world, and it has become central to training as the supply of high-quality human data runs toward the data wall. It is used for instruction tuning, alignment, and especially reasoning, where models generate and filter their own worked solutions, and even in pretraining, where curated high-quality generated data has delivered outsized gains. The famous danger is model collapse: training indiscriminately on unverified model output creates a feedback loop that drifts from reality and erases the rare cases in the data, degrading models across generations. But collapse comes specifically from unverified, recursive generation with no external signal. Curating for quality over volume, filtering generated data through verification, and grounding it in real data or checkable correctness turns synthetic data from a poison into one of the most powerful tools in training. The new information comes from the verification step, which is why it works best where correctness can be checked.

The idea to hold onto is that synthetic data collapses a model when it recycles the model's own unverified errors, but improves the model when a verifier or real-data signal filters those errors out, so the value was never in generating the data but in the check that certifies it, which is why the frontier now depends as much on building good verifiers as on generating good data. Machines can make their own training data, but only useful when something outside the machine can tell the good from the bad.

Common questions

What is synthetic data? Synthetic data is data that is generated artificially rather than collected from the real world, then used to train machine learning models. It can be produced by an AI model generating example text or reasoning, a simulator rendering labelled scenes, generative models creating images, or rules fabricating structured records. It stands in for or supplements real data, and it has become central to modern AI training as the supply of high-quality human data grows scarce. Its usefulness depends heavily on quality and verification, since carelessly generated synthetic data can harm a model while well-curated synthetic data can substantially improve it.

Why is synthetic data used to train AI? Several reasons. As high-quality human data becomes scarce, approaching the so-called data wall, generating data is a way to keep feeding data-hungry training. Synthetic data also fills coverage gaps by producing rare or difficult cases that are uncommon in real data, protects privacy by mimicking the shape of sensitive data without exposing real records, and lowers cost by being cheaper and faster to produce and label than real data. It can be aimed precisely at a model's weaknesses. These advantages have made synthetic data a central part of training, especially for improving reasoning and for specialized capabilities where real examples are limited.

How is synthetic data generated? Through several methods suited to different data types. The most common today is prompting a capable language model to generate example texts, instruction-response pairs, or step-by-step reasoning traces. Simulators using physics and graphics engines generate labelled scenes for robotics and autonomous vehicles, with perfect ground truth. Rule-based generators fabricate structured tabular records, and generative image models produce synthetic pictures. Language-model generation has become dominant for text and reasoning data because a strong model can produce rich, contextually appropriate examples easily, though this is also what makes verifying the generated data essential to avoid degrading the models trained on it.

What is model collapse? Model collapse is the progressive, eventually irreversible degradation that happens when models are trained indiscriminately on unverified model-generated data in a recursive loop. It unfolds in two stages: early collapse, where accumulated errors drift the model away from the true data distribution, and late collapse, where the rare cases in the data, the tails, progressively vanish because models rarely generate what they rarely saw, shrinking diversity each generation. The loop amplifies the model's own mistakes. It appears across generative model families, making it a fundamental risk of naive recursive training, and it is the central warning that any use of synthetic data must address.

Does training AI on AI-generated data always cause collapse? No, and this is the key point. Model collapse is caused specifically by indiscriminate training on unverified model output fed back recursively with no external signal. When synthetic data is curated for quality and diversity, filtered through verification that removes errors, and grounded in real data or checkable correctness, collapse is largely avoided and the data becomes highly useful. Theoretical work confirms that preserving a fraction of real data or injecting a reliable correctness signal keeps training bounded away from collapse. The difference is not human versus synthetic data but whether the synthetic data carries a real signal or merely recycles the model's own errors.

When does synthetic data actually work? It works best when the generated data can be verified. Since checking whether an answer is correct is often far easier than producing a correct answer, generating many candidate outputs and keeping only those that pass a verifier extracts a clean, correct signal, which is why synthetic data is so effective for reasoning tasks in mathematics, code, and logic where correctness is mechanically checkable. It is riskier in open-ended domains where "correct" is fuzzy and no clean verifier exists. In short, synthetic data works when something outside the model, a verifier, a tool, ground truth, or retained real data, can reliably separate the good generated examples from the bad.

Is synthetic data the answer to the data wall? It is a major part of the answer, but with an important qualification. Synthetic data lets training continue past the point where human data runs short, and it powers much of the recent progress in reasoning and specialized capabilities. But it does not create knowledge from nothing; its value comes from a verification or real-data signal that certifies which generated examples are good. So the future of training past the data wall depends less on simply generating more data than on building better ways to verify it. Synthetic data extends the runway considerably, but only when paired with the checks that keep it grounded in reality.