Agent Evaluation
Measuring whether an agent actually works — much harder than evaluating a model, and the reason agent demos and agent products are different things.
When not to use it
- (The question is which evaluation to distrust.)*
- Single-run demos. Agents are stochastic. One success is not a measurement.
- Per-step accuracy as end-to-end. It's the number that looks fine and doesn't mean anything.
- Simulation results as production evidence. The sandbox is not the world, and it flatters.
- LLM-judge scores on reasoning quality. Known biases, and a model grading a model is circular.
Reach for something else instead
- Your own thirty tasks — real, from your use case, with checkable outcomes. Worth more than every leaderboard.
- Executable success criteria — tests that pass. If you can arrange it, arrange it.
- Human review of trajectories — expensive, and the only honest way to judge process.
- Shadow deployment — run alongside a human, compare. Slow, and it's the real answer.
Sources & further reading
- Jimenez et al. (2023), SWE-bench: Can Language Models Resolve Real-World GitHub Issues? — executable success on real repositories; the honest benchmark.
- Liu et al. (2023), AgentBench: Evaluating LLMs as Agents — the multi-environment attempt, and its measurement difficulties.
- Zheng et al. (2023), Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena — the biases in using a model to grade a model.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Not measuring variance. Ten runs of the same task tells you what one run cannot.
- Reporting outcome without cost or step count. An agent succeeding in forty steps is failing.
- Believing benchmark numbers on public benchmarks. They're in training data now.
- Using an LLM judge for trajectory quality without acknowledging the circularity.
- Treating a demo as evidence. It was chosen; your task wasn't.
At a glance
FieldAI Agents
Harder than model eval becauseside effects, path-dependence, no trajectory ground truth
The killer numberper-step accuracy to the power of steps
Gold standardexecutable success (SWE-bench)
Rarely reportedvariance across runs
DifficultyIntermediate
Flashcards for this concept
Question
—
Answer
—
1 / 4
Often compared with
Outcome vs. trajectory evaluation — one asks whether the job got done, the other whether it was done sensibly. Outcome is measurable; trajectory is what you actually care about.