Error Asymmetry
When being wrong in one direction costs far more than being wrong in the other, which makes a single accuracy figure close to meaningless.
Reviewed July 16, 2026Stable
When not to use it
- Where the costs genuinely are symmetric, which is rare and worth verifying rather than assuming in either direction.
- As a reason to ignore accuracy entirely; it remains necessary and is not sufficient.
- Where the operating point has already been set from a stated cost analysis, which is the practice this concept exists to encourage.
Reach for something else instead
- Capped constraint — fix a maximum on the costly error rate and minimise the other subject to it, avoiding the need to convert costs into a common unit.
- Subgroup reporting — publish error rates by population, which reveals concentration that an aggregate conceals.
- -
Read more on the blog
- Robots weed ten million acres. They still cannot pick.Agricultural robotics has a clean split between what scales and what does not, and the line falls exactly where the thesis of this territory predicts.
- Williams v Detroit: the match was not the failureThe first publicly reported wrongful arrest from a face recognition match. The settlement's remedy is procedural, and it names a failure that has nothing to do with model accuracy.
- The Dutch benefits scandal: the rule, not the modelAround 26,000 families were wrongly accused of fraud and a government resigned. The parliamentary inquiry did not blame the algorithm. What it found is more useful and less quoted.
- The sepsis model caught 7% of what clinicians missedA sepsis warning system ran at hundreds of US hospitals before anyone outside the vendor validated it. The external check found the number that matters is not the one being reported.
Further reading
- Kleinberg et al. (2016), Inherent Trade-Offs in the Fair Determination of Risk Scores — why error rates cannot be equalised across groups while calibration holds.
- Wong et al. (2021), External Validation of a Widely Implemented Proprietary Sepsis Prediction Model — alert burden against omission, where only one side leaves a record.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Comparing systems on a single balanced metric when their deployments have different cost structures.
- Reporting aggregate error rates where the errors concentrate on an identifiable subgroup.
- Assuming an unmeasured error is a rare one, when it may simply leave no artefact.
At a glance
FieldFoundations
Core pointone accuracy figure hides a cost weighting
Three questionsreversible, borne by whom, visible
Formal remedycost matrix or capped constraint
Common trapthe costlier error is the less measurable one
DifficultyBeginner → Intermediate
Where this sits
A starting point. Nothing needs to come before it.
0Levelstarting point
0Needs firstnothing
0Opens upnothing further
1Areastays here
Computed from the prerequisite graph, not assigned. How this works