Voice Cloning
Copying a specific person's voice from a short sample — technically impressive, ethically unresolved, and already being used against people.
When not to use it
- Without documented, specific, revocable consent. Not a legal opinion — a description of the only version of this that survives scrutiny.
- For anyone deceased, without the estate. The "they'd have wanted it" argument has no limiting principle.
- For anything where the voice authenticates. Voice as a security factor is over. Treat it as over.
- When a generic voice would do. If you don't need this person, don't clone a person.
Reach for something else instead
- Licensed synthetic voices — professionally recorded, consented, commercially clear.
- Generic TTS — for most applications, nobody needed a specific human's voice.
- Recorded audio — if the person is available and the content is fixed.
- Voice conversion with consent — the same technology, with the consent problem solved rather than ignored.
Sources & further reading
- Wang et al. (2023), Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers — VALL-E; cloning from three seconds by treating audio as tokens.
- Jia et al. (2018), Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis — the speaker-embedding approach that separated identity from content.
- San Roman et al. (2024), Proactive Detection of Voice Cloning with Localized Watermarking — watermarking as the defence, and its dependence on generator cooperation.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Treating consent as a checkbox at upload. Someone will upload a voice they don't own, and "they clicked yes" is not a defence.
- Relying on detection classifiers. They're trained on the artefacts of known generators and fail on new ones.
- Assuming watermarking closes the gap. It works when the generator cooperates; open-weight models don't have to.
- Still using voice for authentication. That was already unsafe and is now indefensible.
- Building the product before the consent process. The consent process is the product.
At a glance
FieldSpeech & Audio
Audio neededseconds, for zero-shot
Mechanismspeaker embedding or audio-token in-context learning
Detectionbrittle
Watermarkingworks only if the generator cooperates
Statuscapability solved, ethics unresolved
DifficultyIntermediate
Flashcards for this concept
Question
—
Answer
—
1 / 4
Often compared with
Voice cloning vs. TTS — the same machinery, aimed at a specific person. The technical difference is a speaker embedding; the practical difference is consent.