AI Detector
A tool that claims to tell whether text was written by AI — and the peer-reviewed finding that they systematically flag non-native English speakers as machines.
When not to use it
- On people. Students, applicants, employees — the false positives are systematic, they concentrate on non-native speakers, and the consequences land on individuals.
- As evidence of anything. The output is a number with no established error rate on your population.
- To catch deliberate misuse. Paraphrasing defeats it, so it only catches people who weren't hiding.
Reach for something else instead
- Watermarking attacks provenance instead of detection — tractable, and only works where the generator cooperates.
- Content credentials (C2PA-style signing) prove what a thing is rather than guessing what it isn't.
- Assessment design is the real answer in education: oral defence, drafts, in-class work. Change the task, not the surveillance.
Sources & further reading
- Liang et al. (2023), GPT Detectors Are Biased Against Non-Native English Writers — Patterns; over half of non-native essays misclassified as AI, and simple prompting defeats detection.
- Sadasivan et al. (2023), Can AI-Generated Text Be Reliably Detected? — the theoretical bound; detection degrades toward random as models improve.
- Krishna et al. (2023), Paraphrasing Evades Detectors of AI-Generated Text — paraphrasing defeats every detector tested, including watermarking and retrieval defences.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Treating a percentage as a probability. It is a classifier score with no calibration on your population, and the paper says its errors are systematic.
- Assuming false positives are random. They aren't — they concentrate on non-native English writers, which turns the tool into a discrimination mechanism with a number attached.
- Believing better detectors are coming. The theoretical result runs the other way: as models improve, detection provably degrades.