Agent Evaluation
Measuring whether an agent actually works — much harder than evaluating a model, and the reason agent demos and agent products are different things.
Reviewed July 12, 2026Stable
When not to use it
- (The question is which evaluation to distrust.)*
- Single-run demos. Agents are stochastic. One success is not a measurement.
- Per-step accuracy as end-to-end. It's the number that looks fine and doesn't mean anything.
- Simulation results as production evidence. The sandbox is not the world, and it flatters.
- LLM-judge scores on reasoning quality. Known biases, and a model grading a model is circular.
Reach for something else instead
- Your own thirty tasks — real, from your use case, with checkable outcomes. Worth more than every leaderboard.
- Executable success criteria — tests that pass. If you can arrange it, arrange it.
- Human review of trajectories — expensive, and the only honest way to judge process.
- Shadow deployment — run alongside a human, compare. Slow, and it's the real answer.
Read more on the blog
- The Tempe crash: it saw her for 5.6 secondsThe NTSB found the system detected the pedestrian 5.6 seconds before impact, reclassified her repeatedly, and could not label a person outside a crosswalk. Every safeguard had been disabled for ride smoothness.
- What to measure before you deploy an agentAgents that succeed 60% of the time on a single run succeed 25% of the time across eight. Task completion is the metric everyone reports and it is wrong in three separate ways.
- Your agent works in the demo because the demo is the easy caseA demo is not a small production system. It is the survivor of a search process designed to find something that works, which makes it evidence about your search rather than about the system.
- Your agent returned 200 OK and did the wrong thingOrdinary software tells you when it breaks. AI systems return well-formed, confident, wrong answers with a success status. That one property is why observability for AI is a different discipline rather than the old one pointed at new infrastructure.
Further reading
- Jimenez et al. (2023), SWE-bench: Can Language Models Resolve Real-World GitHub Issues? — executable success on real repositories; the honest benchmark.
- Liu et al. (2023), AgentBench: Evaluating LLMs as Agents — the multi-environment attempt, and its measurement difficulties.
- Zheng et al. (2023), Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena — the biases in using a model to grade a model.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Not measuring variance. Ten runs of the same task tells you what one run cannot.
- Reporting outcome without cost or step count. An agent succeeding in forty steps is failing.
- Believing benchmark numbers on public benchmarks. They're in training data now.
- Using an LLM judge for trajectory quality without acknowledging the circularity.
- Treating a demo as evidence. It was chosen; your task wasn't.
At a glance
FieldAI Agents
Harder than model eval becauseside effects, path-dependence, no trajectory ground truth
The killer numberper-step accuracy to the power of steps
Gold standardexecutable success (SWE-bench)
Rarely reportedvariance across runs
DifficultyIntermediate
Often compared with
Outcome vs. trajectory evaluation — one asks whether the job got done, the other whether it was done sensibly. Outcome is measurable; trajectory is what you actually care about.
Where this sits
29 concepts come first. Understanding it opens up 1 more.
12Levelsteps in
29Needs firstconcepts
1Opens up0% of 310
2Areascrosses fields
Computed from the prerequisite graph, not assigned. How this works