How AI generates images: from noise to a picture

Type "a fox reading a newspaper in a Paris café, oil painting" into an image generator and, seconds later. You have a picture that has never existed anywhere, coherent, detailed, stylistically consistent, invented on demand. This is one of the most visible and striking capabilities of modern AI, and unlike much of the field, the core idea behind it is both different from how language models work and, once you see it, strangely neat.

Most of our other explainers are about language models, systems that predict text. Image generation runs on a largely different engine, and this piece opens that whole side of AI: what a diffusion model actually is, why "learning to remove noise" turns out to be a way to create, how a text prompt steers the process toward your fox-in-a-café, and the live 2026 rivalry between diffusion and a newer approach borrowed from language models. By the end, the magic of typing a sentence and getting a picture should resolve into something you understand.

The counterintuitive core idea: learn to denoise

Here is the central trick, and it's clever. To teach a model to create images, you first teach it to destroy them, and then run the destruction backwards.

Start with the training process. Take a real image and add a tiny bit of random noise (visual static). Add a bit more. And more, over many small steps, until the image is pure noise, indistinguishable from television snow, all trace of the original gone. This is the forward process, and it involves no learning at all; it's just progressively corrupting data into randomness. Do this to millions of images.

Now the clever part. Train a neural network to do the reverse: given a slightly noisy image, predict and remove a small amount of noise, nudging it back toward a clean image. The model learns this one narrow skill, "undo a little noise", from millions of examples of noisy-to-slightly-less-noisy pairs. That's the entire training objective. It never learns to "draw"; it learns to denoise, one small step at a time.

The payoff comes at generation. Once the model is a skilled denoiser, you hand it something it has never seen in training: pure random noise, straight from a random number generator. Then you ask it to denoise, remove a little noise, then denoise the result again, and again, hundreds of times. And here is the clever part: because the model learned what real images look like as it removed noise from real training images, when you run it on random noise it hallucinates structure into the static, step by step, until a coherent image emerges that was never there. It's not recalling a stored picture; it's sculpting one out of noise, guided only by its trained sense of what images should look like. The reason this works in small steps rather than one leap is that each denoising step is a simple problem the network can solve reliably, turning an impossible task (noise → image) into hundreds of easy ones.

The analogy that captures it: a diffusion model is like a photo restorer who trained on millions of damaged photos until they could reconstruct a clear image from almost any corruption, then you hand them pure static and their instinct to "restore" invents a plausible photo from nothing. Denoising, pushed to its limit, becomes generation.

How a text prompt steers it

Denoising random noise produces a random image. To get your fox-in-a-café, the process needs steering, and this is where the text prompt enters, connecting image generation back to the language-model world.

Your prompt is first converted into an embedding, a numerical representation of its meaning, produced by a text encoder much like the ones language models use. Then, at every denoising step, that text embedding is fed into the denoising network as a condition, via attention, the same mechanism from transformers. The effect is that the model isn't just asked "remove noise to make this look like any real image," but "remove noise to make this look like a real image that matches this text." The prompt biases every step of the sculpting toward images consistent with your words. Over hundreds of steps, that steady pressure accumulates: the noise resolves not into a random scene but into the fox, the newspaper, the café, the oil-painting style, because each step was nudged toward matching the prompt.

This is the key thing to understand about how these models relate to your words: the model does not draw from your prompt directly. It has learned a vast visual space from training, and the prompt merely steers which region of that space the denoising heads toward. The prompt sets constraints; the model fills in every detail you didn't specify from patterns it learned. That's why the same prompt yields different images each run (different starting noise), why unmentioned details vary wildly, and why prompting is a matter of guidance rather than instruction, you're pointing at a region of visual possibility, not dictating pixels.

Working in a compressed space: why it's fast enough to use

One more piece explains how this runs in seconds rather than hours. Doing all that denoising directly on full-resolution pixels, millions of them, would be enormously expensive. So modern systems (the "latent diffusion" that powers the well-known image generators) do the denoising in a compressed latent space instead.

The trick uses a variational autoencoder: an encoder compresses images into a much smaller latent representation (capturing the essence in far fewer numbers), the whole diffusion denoising process happens in that compact space, and then a decoder expands the final denoised latent back into a full-resolution image. Because the expensive denoising loop runs on the small representation rather than raw pixels, generation becomes fast and cheap enough for everyday use. This is a quietly key engineering move, it's the difference between a research curiosity and a tool millions use, and it's why text-to-image generation became a consumer product rather than a supercomputer demo.

Why diffusion beat the previous champion

Diffusion isn't the first approach to AI image generation, and it's worth knowing what it displaced, because the contrast illuminates why it won. The previous state of the art was the GAN (generative adversarial network), which pits two networks against each other, a generator making fakes and a discriminator trying to catch them, until the fakes are convincing. GANs produced the first realistic AI faces and were dominant for years.

Diffusion overtook them for a few reasons that matter. GANs are notoriously unstable to train, the two-network game can collapse in frustrating ways, while diffusion's "just learn to denoise" objective is far more stable and reliable. Diffusion also tends to produce more diverse outputs (GANs can get stuck generating variations of the same thing) and scales more gracefully to high quality. The trade is speed: a GAN generates in a single pass, while diffusion needs many denoising steps, making it slower. But the stability, diversity, and quality won out, and by the mid-2020s diffusion had become the de facto approach for turning text into images.

The 2026 rivalry: diffusion vs. the language-model approach

This is where the story gets current, because image generation has not settled the way language modeling has, and 2026 features a genuine architectural contest.

The challenger comes straight from the language-model world: autoregressive image generation. The idea is to treat an image the way a language model treats a sentence, convert the image into a sequence of discrete visual "tokens" (via an encoder that compresses patches into codes), then generate those tokens one at a time, in order, each conditioned on the previous ones, exactly like a language model predicts the next word. If you understand how an LLM writes text token by token, you already understand this: same next-token-prediction machinery, pointed at image tokens instead of words.

Each approach has real advantages, which is why neither has won outright. Diffusion excels at visual fidelity, models the continuous nature of pixels naturally without needing tokenization, and can generate all parts of an image in parallel and even "fix" things mid-process because it keeps refining the whole canvas. Autoregressive models are often easier to train, scale well with sequence length, and, crucially, slot naturally into multimodal systems, because if images are just tokens, a single model can fluidly handle text and images in the same token stream. That last point is why the approach is gaining momentum: as AI moves toward unified models that see, read, and generate across modalities, treating images like language becomes attractive.

Video is where this contest is fiercest and least settled. Generating video means generating many frames that must be both individually sharp and temporally coherent (no flickering, consistent objects across frames), and it's hard. Diffusion gives strong per-frame quality and temporal consistency; autoregressive approaches (generate frames in sequence, each conditioned on the last, like the video is a story unfolding) enable streaming and interactivity but can accumulate errors over long clips. So the frontier is increasingly hybrid, using diffusion for the fine visual detail within chunks and autoregressive generation for coherence across time. Unlike language modeling, which converged firmly on one architecture, visual generation in 2026 is still an open contest, and that's part of what makes it the field's most dynamic frontier.

The short version

AI generates images primarily through diffusion: a model is trained to remove noise from images, one small step at a time, by learning from millions of examples of images being progressively corrupted into static. To create, you hand the trained model pure random noise and let it denoise repeatedly, and it sculpts a coherent image out of the static, guided by its learned sense of what real images look like. A text prompt steers every denoising step (via the same attention mechanism language models use) toward images matching your words, and the whole process runs in a compressed latent space to make it fast. Diffusion displaced the earlier GAN approach on stability and quality, and in 2026 it's contesting the frontier, especially in video, with an autoregressive approach borrowed straight from language models.

an AI image generator doesn't paint, it denoises, turning random static into a picture one small step at a time, steered by your words toward a region of the vast visual space it learned. The magic of typing a sentence and getting an image is the magic of a very good denoiser, handed noise and a nudge, hallucinating structure into chaos until a picture appears.

Common questions

How does AI image generation actually work? Most AI image generators use diffusion models. During training, the model learns to remove noise from images by studying millions of examples of pictures being progressively corrupted into static. To generate a new image. You give the trained model pure random noise and it denoises it step by step, hundreds of small steps, sculpting a coherent image out of the static, guided by its learned sense of what real images look like. A text prompt steers each step toward images matching your words. The model doesn't retrieve or copy; it invents a new image from noise.

What is a diffusion model? A diffusion model is a generative AI model that creates images (or audio, or video) by learning to reverse a noising process. It's trained by adding noise to real data step by step until it's pure static, then learning to undo each step. Once trained, it starts from random noise and iteratively removes noise to produce entirely new data. The name comes from the physics-like process of gradually diffusing data into noise and then reversing it. Diffusion models power most modern image generators.

Why does AI image generation start with noise? Because the model was trained to remove noise, not to draw from scratch. Its one skill is turning a noisy image into a slightly cleaner one. Random noise is simply the starting point that this skill can operate on: by repeatedly denoising pure static, the model progressively imposes the structure of real images it learned during training, until a coherent picture emerges. Starting from noise also means every generation is different, since a different random starting point leads to a different final image.

How does the text prompt control the image? Your prompt is converted into a numerical embedding representing its meaning, and that embedding is fed into the model at every denoising step through attention (the same mechanism transformers use). This biases each step toward producing an image that matches your words. Importantly, the prompt steers rather than dictates, it points the denoising toward a region of the visual space the model learned, and the model fills in all the details you didn't specify from training patterns. That's why the same prompt produces different images and unmentioned details vary.

What's the difference between diffusion models and GANs? GANs (generative adversarial networks) generate images in a single pass using two competing networks, a generator and a discriminator, and were the previous state of the art. Diffusion models generate through many denoising steps. Diffusion largely replaced GANs for image generation because it's far more stable to train (GANs can collapse unpredictably), produces more diverse outputs, and scales better to high quality. The trade-off is speed: GANs generate in one pass while diffusion needs many steps, though various techniques have narrowed that gap.

What is autoregressive image generation? It's an alternative to diffusion that treats an image like a language model treats text. The image is converted into a sequence of discrete visual tokens, and the model generates those tokens one at a time, each conditioned on the previous ones, exactly like an LLM predicts the next word. Its advantages are easier training, good scaling, and natural fit with multimodal systems that handle text and images in one token stream. In 2026 it's a serious rival to diffusion, especially as AI moves toward unified models, though diffusion still leads on pure visual fidelity. Video generation increasingly uses hybrids of both.

Why do AI images get hands and text wrong? Because image models learn statistical patterns of what images look like rather than the underlying rules of anatomy or spelling. Hands are hard because they vary enormously in pose and are often partly hidden in training images, so the model learns a fuzzy sense of hand-like that frequently produces the wrong number of fingers. Text is hard because the model treats letters as visual shapes to reproduce, not symbols with meaning, so it renders plausible-looking glyphs that do not spell real words. Both are improving as models scale, but they remain classic tells because they demand precise structure the model was never explicitly taught.