Diffusion Model
How most AI image tools work — starting from random noise and removing it step by step, guided by a prompt, until a picture appears.
When not to use it
- When you need the same output twice. Diffusion is stochastic by design. For deterministic assets, generate once and store the file.
- For text inside images, precise counts, or exact layouts. These are known weak spots; the model is painting what text looks like, not typesetting it.
- Where provenance matters. If you can't say where the training data or the output came from, that's a legal and editorial question before it's a technical one.
Reach for something else instead
- Autoregressive image models and GANs each trade differently on speed, diversity, and control; GANs are still faster at inference for narrow domains.
- Templates and design tools when you need exact, repeatable, brand-correct output — which is most commercial work.
- Stock or commissioned imagery when licensing clarity is worth more than novelty.
Sources & further reading
- Ho, Jain & Abbeel (2020), Denoising Diffusion Probabilistic Models — the paper that made diffusion work.
- Rombach et al. (2022), High-Resolution Image Synthesis with Latent Diffusion Models — Stable Diffusion, and the move to latent space that made it runnable on consumer hardware.
- Song, Meng & Ermon (2020), Denoising Diffusion Implicit Models — fewer steps, and the speed/quality trade-off you actually tune.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Prompting harder to fix a structural failure. If the model can't do hands or text, more adjectives won't help — inpainting or a different tool will.
- Ignoring the step-count/quality trade-off, then complaining about latency. Fewer steps is often nearly as good and much faster.
- Treating the seed as irrelevant. It's the one lever that makes results reproducible enough to iterate on.
At a glance
FieldGenerative AI
Core ideadenoise from static to image
Trade-offquality vs. number of steps
Key controlssteps, guidance, seed
DifficultyIntermediate
Flashcards for this concept
Question
—
Answer
—
1 / 4
Often compared with
Diffusion vs. GAN — iterative denoising vs. a generator trained against a discriminator.