Benchmark
A standard test used to compare AI systems — indispensable for progress, and routinely mistaken for a measure of the thing it approximates.
When not to use it
- To decide whether a model works for your task. It can't tell you that. Thirty of your own examples can.
- When the benchmark is saturated. If everything scores 92%, the differences are noise dressed as signal.
- As evidence of intelligence, understanding, or reasoning. It's evidence of performance on the benchmark. The rest is inference and it's contested.
- When the benchmark predates the model by years. Contamination is likely and the score is closer to a memorisation test.
Reach for something else instead
- Your own evaluation set — thirty real examples. The most valuable artefact in any AI project.
- Executable benchmarks (SWE-bench style) — objective, harder to game, closer to real work.
- Human preference evaluation — catches what automated tests miss; brings its own confounds.
- A/B testing in production — the only measure of whether users got what they needed.
Sources & further reading
- Chollet (2019), On the Measure of Intelligence — benchmarks measure skill, not intelligence; the case for efficiency of acquisition instead.
- Hendrycks et al. (2021), Measuring Massive Multitask Language Understanding — MMLU, and worth reading for what its authors claim it measures versus how it gets cited.
- Sainz et al. (2023), NLP Evaluation in Trouble: On the Need to Measure LLM Data Contamination for each Benchmark — the contamination problem stated plainly.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Reading a benchmark score as a general capability claim. It's a claim about one test.
- Comparing scores across differently-configured runs. Prompt format, few-shot count and parsing all move numbers by several points.
- Ignoring the saturation point — celebrating 94% vs. 92% on a benchmark where both are noise.
- Assuming decontamination worked. It's best-effort substring matching against an unauditable corpus.
- Building your product around a leaderboard rank rather than your own thirty examples.
At a glance
FieldFoundations
What it measuresperformance on itself
What it's read asgeneral capability
Main threatscontamination, saturation, Goodhart
Best alternativethirty of your own examples
DifficultyBeginner
Flashcards for this concept
Question
—
Answer
—
1 / 4
Often compared with
Benchmark vs. your own eval set — one tells you the model is in the right league; the other tells you whether it does your job. Only one of those is a decision.