Chain-of-Thought
Getting a model to reason step by step before answering — which dramatically improves its performance on hard problems.
When not to use it
- On simple tasks. It adds tokens, latency, and cost for no gain — and on easy questions it can talk the model out of a correct first instinct.
- As an explanation of the model's actual process. The stated reasoning is generated text, not a transcript of computation. It can be plausible and unrelated to how the answer was reached.
- When you need short answers. Reasoning that leaks into the output is a formatting bug for most product surfaces.
Reach for something else instead
- Few-shot examples often get the same lift with fewer tokens.
- Tools — for arithmetic or lookups, let the model call a calculator or a database rather than reason through it.
- Decomposition in your code — separate prompts per step gives you control, checkpoints, and debuggability.
Sources & further reading
- Wei et al. (2022), Chain-of-Thought Prompting Elicits Reasoning in Large Language Models — the original.
- Kojima et al. (2022), Large Language Models are Zero-Shot Reasoners — the step-by-step result.
- Turpin et al. (2023), Language Models Don't Always Say What They Think — stated reasoning can be plausible and unfaithful. Read this before trusting a chain.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Trusting the reasoning because it sounds rigorous. Faithfulness of stated reasoning is an open research problem, not a solved one.
- Using it everywhere by default. Measure it; on many tasks it costs more and helps nothing.
- Showing the chain to end users, who reasonably read it as the system's real thinking.
At a glance
FieldLanguage & LLMs
Core ideareason step by step before answering
Benefitbig accuracy gains on hard tasks
Caveatreasoning may not be faithful
DifficultyBeginner → Intermediate
Flashcards for this concept
Question
—
Answer
—
1 / 4