Deepfake
Synthetic media of a real person doing something they didn't — where detection is losing, the harm is already overwhelmingly to private individuals, and it isn't mostly about elections.
When not to use it
- (It's a harm, not a tool. The question is what defences to trust.)*
- Detectors, as a reliable defence. They don't generalise across methods and compression destroys the signal.
- A published detector, at all. It becomes the attacker's training target.
- Detection as the strategy. The distributions are converging by design. This race has a known ending.
- Assuming it's mainly a political problem. The measured harm is overwhelmingly non-consensual imagery of private women.
Reach for something else instead
- Provenance (C2PA, content credentials) — sign at capture, track edits. The only approach that can work.
- A family code word — defeats voice-clone fraud entirely, costs nothing.
- Platform policy and legal remedy — where the actual harm is, this is where the action is.
- Institutional verification — chains of custody, as before photography was trusted.
Sources & further reading
- Ajder et al. (2019), The State of Deepfakes — the measurement; the overwhelming majority is non-consensual sexual content targeting women.
- Chesney & Citron (2019), Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security — the liar's dividend; the second-order harm that matters most.
- Rössler et al. (2019), FaceForensics++: Learning to Detect Manipulated Facial Images — the detection benchmark, and why detectors don't generalise.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Framing it as an election problem. That's the coverage, not the harm.
- Betting on detection. The generator's objective is literally to defeat it.
- Publishing your detector, which trains the next generator.
- Missing the liar's dividend. The damage is truths becoming deniable, and it's already done.
At a glance
FieldSafety & Ethics
Where the harm actually isnon-consensual sexual imagery of private women, by a wide margin
What changedcost, not capability
Why detection losesthe distributions converge by design
The second-order harmthe liar's dividend: real video loses its force
The only workable defenceprovenance
DifficultyBeginner
Flashcards for this concept
Question
—
Answer
—
1 / 4
Often compared with
Detection vs. provenance — one asks "does this look fake," which is becoming unanswerable; the other asks "is this signed," which is checkable.