Synthetic Data
Training on data a model generated — increasingly standard, genuinely useful, and carrying a failure mode with a Nature paper attached.
Reviewed July 13, 2026Stable
When not to use it
- As a replacement for real data. That's the recursive setup that collapses. Accumulate, don't replace.
- Without a verifier, at scale. Unfiltered synthetic data is the generator's beliefs, errors included.
- For rare events and tails. Those are the first thing the generator under-represents — exactly what you needed.
- Where the distribution matters and can't be checked. Judgement, taste, strategy. No filter, no signal.
Reach for something else instead
- Real data — the thing this substitutes for, with the caveats.
- Data augmentation — transformations of real data. Lower risk, less coverage.
- Rejection sampling with a verifier — the version that reliably works.
- Transfer learning — use a model that already saw real data.
Read more on the blog
- What is synthetic data? Training AI on AI-made dataFaced with running out of human text to train on, AI labs increasingly train models on data the models generate themselves. This is synthetic data, and it comes with a famous warning called model collapse. The resolution of that tension, that verification and curation are what separate collapse from improvement, is the whole story and one of the most important ideas in how modern AI is built.
- How do we measure AI progress? The benchmark problemEvery AI model launches with a table of benchmark scores that look like objective proof of progress. A growing body of evidence says those numbers are far less trustworthy than they appear, because benchmarks saturate, leak into training data, and reward the wrong thing. This is the evaluation problem, and it has quietly become one of the hardest parts of AI.
- How AI models are trained: from raw text to a system that helpsA language model isn't programmed, it's grown, in stages, from a firehose of text into a system that answers helpfully. The full modern pipeline: pretraining, supervised fine-tuning, preference alignment, and the reasoning training that defines 2026, what each stage does, why none can be skipped, and how the recipe changed.
Further reading
- Shumailov et al. (2024), AI models collapse when trained on recursively generated data — Nature; the mechanism, and the tails go first.
- Gerstgrasser et al. (2024), Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data — the correction: accumulate, don't replace.
- Wang et al. (2022), Self-Instruct: Aligning Language Models with Self-Generated Instructions — how most open instruction data actually exists.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Reading model collapse as "synthetic data is doomed." Accumulating real plus synthetic avoids it; only replacement fails.
- Generating without filtering, then wondering why the model inherited the generator's errors.
- Expecting synthetic data to cover rare cases. Finite sampling loses the tails first — that's the mechanism.
- Assuming your web crawl is human-written. Increasingly it isn't, and nothing labels it.
At a glance
FieldMachine Learning
The failure modemodel collapse; the tails go first, then the variance
Whyfinite resampling under-represents rare events, compounding each generation
The fixaccumulate real + synthetic; don't replace
Where it genuinely worksverifiable domains with a filter
DifficultyIntermediate
Often compared with
Synthetic data with a verifier vs. without — one adds external signal and works indefinitely; the other recycles the generator's beliefs and degrades.
Where this sits
1 concept come first. Understanding it opens up 1 more.
1Levelsteps in
1Needs firstconcepts
1Opens up0% of 310
1Areastays here
Learn these firstTraining Data
Computed from the prerequisite graph, not assigned. How this works