Test-Time Compute
Spending more compute when the model answers rather than when it trains — the scaling axis the field found after the first one got expensive.
Reviewed July 14, 2026Moving
When not to use it
- Tasks with no verifier and no majority to take. Extra samples give you more text and no way to choose, which is spending without buying.
- Latency-critical paths. This axis converts compute into time by construction.
- Anything where a cheaper model already saturates the task. Test-time compute is a multiplier on a gap; with no gap there is nothing to multiply.
Reach for something else instead
- A better base model is the training-time lever, and for high-volume products it is usually the cheaper one, amortised.
- Retrieval fixes the knowledge failures people often try to fix with more thinking. The model wasn't reasoning badly; it didn't have the fact.
- A verifier plus a small model frequently beats a large model thinking hard, because the check contributes more than the thinking.
Read more on the blog
- What AI cannot do, no matter how capable it getsFrontier models now rank just behind specialist verification tools at deciding whether programs terminate, which is supposedly undecidable. Both facts are true, and the reason they coexist is the thing most people get wrong.
- How reasoning models work: AI that thinks before answeringIn 2025 a new kind of model arrived: one that pauses to think, working through a problem on internal scratch paper before answering. Reasoning models like o1, o3, and DeepSeek-R1 trade speed for accuracy on hard problems. Here is what they actually do, how they learned to do it, and when the extra thinking is worth it.
- AI scaling laws: why bigger wins, and whether it's endingOne empirical discovery explains most of the last six years of AI and the hundreds of billions spent on it: model performance improves in a smooth, predictable way as you add size, data, and compute. Here is what scaling laws are, why they reshaped the field, the Chinchilla correction, and the live 2026 question of whether pure scaling is running out.
Further reading
- Snell et al. (2024), Scaling LLM Test-Time Compute Optimally Can Be More Effective Than Scaling Model Parameters — the compute-allocation result.
- Brown et al. (2024), Large Language Monkeys: Scaling Inference Compute with Repeated Sampling — coverage rises log-linearly with samples; verification is the constraint.
- Wang et al. (2023), Self-Consistency Improves Chain of Thought Reasoning in Language Models — the simplest method that works.
- Lightman et al. (2023), Let's Verify Step by Step — process supervision, the verifier side of the trade.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Assuming more samples means better answers. Coverage rises; selection doesn't come free, and without a verifier you're picking the most plausible-looking, which is not the same thing.
- Spending the same budget on every question. The optimal allocation depends on difficulty — uniform spend wastes most of it on questions that were easy.
- Treating it as a substitute for training-time scale. They're different axes with different economics, and the right split depends on your query volume, not on which is fashionable.
At a glance
FieldFoundations
The tradecompute at answer time vs. training time
Bottleneckverification, not generation
Made famous byreasoning models, 2024–25
DifficultyIntermediate
Where this sits
A destination. 31 concepts lead here, and nothing in the corpus depends on it.
15Levelsteps in
31Needs firstconcepts
0Opens upnothing further
1Areastays here
Computed from the prerequisite graph, not assigned. How this works