Bayes' Theorem
The rule for updating a belief when new evidence arrives — the mathematical backbone of learning from data, and a genuine fix for how badly human intuition handles probability.
When not to use it
- When you genuinely have no basis for a prior and are unwilling to state one — though "I don't know" is itself a prior (a flat one).
- When the naive independence assumptions of a specific Bayesian model are badly violated and you ignore it.
- As a substitute for causal reasoning — Bayes updates on correlation-carrying evidence, not causal structure.
Reach for something else instead
- Frequentist inference (p-values, confidence intervals) for the same problems under a different philosophy.
- Point-estimate methods when uncertainty genuinely doesn't matter and you just need a best guess.
Bayes' theorem: why a positive test can still mean low risk.
When a disease is rare, even a small false-positive rate among the huge healthy majority produces more false alarms than true cases. The tiny green slice is the true positives; the larger amber slice is false positives — so a positive result, before you know more, is more likely a false alarm. That's the base-rate effect the theorem quantifies.
Worked example
The classic trap, with numbers. A disease affects 1 in 1,000 people. A test is 99% accurate both ways. You test positive. What's the chance you're actually sick? Intuition says ~99%. Watch.
Imagine 100,000 people:
→ test positive (99%): 99 true positives
Actually healthy: 99,900 people
→ test positive (1% error): 999 false positives
So of everyone who tests positive, how many are truly sick?
= 99 / 1,098 = 0.09 ← just 9%
A 99%-accurate test, and a positive result still means only a 9% chance of disease — because the healthy group is so much larger that its 1% error produces ten times more false alarms than there are true cases. That gap between 99% and 9% is the base-rate fallacy, and it's why the theorem exists.
Sources & further reading
- Bayes (1763), An Essay towards solving a Problem in the Doctrine of Chances — the original, published posthumously.
- Gigerenzer & Hoffrage (1995), How to Improve Bayesian Reasoning Without Instruction — the natural-frequency framing that makes it intuitive.
- Bishop, Pattern Recognition and Machine Learning — the Bayesian foundations of modern ML.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Ignoring the prior / base rate — the rare-disease error, the most common probability mistake there is.
- Confusing P(B|A) with P(A|B) — the "prosecutor's fallacy," which the theorem exists to prevent.
- Treating the posterior as certainty rather than an updated belief still carrying uncertainty.