Field
Speech & Audio
Machines that hear and speak — the oldest AI dream, and the one with the widest gap between demo and reality.
Speech was supposed to be the natural interface. Talk to the computer, it understands, it answers. Decades of work went into it, and the current state is stranger than either the optimism or the cynicism predicted.
Speech recognition is close to solved for one person speaking clearly into a decent microphone in a quiet room — better than human typists, and cheap. It falls apart on almost everything else: two people talking at once, a strong accent, a café, a vocabulary it has never heard. The reported accuracy figures describe the easy case, and the gap between them and your recording is where every speech project lives.
Text-to-speech went the other way. Naturalness — the thing that made synthetic voices sound robotic for forty years — is essentially solved. What remains is prosody: knowing which word to stress. That information isn't in the text, so a model reading a sentence in isolation is guessing at intent, which is why synthetic speech now sounds subtly uninvolved rather than obviously wrong.
The uncomfortable entries are here too. Voice cloning needs seconds of audio, detection is losing an arms race it structurally cannot win, and the practical consequence is that a voice is no longer evidence of identity. Speaker diarization — working out who spoke when — is the half of transcription that quietly gets it wrong. Music generation is blocked by provenance, not quality.
Start with Speech Recognition — and note that the highest-leverage fix in the whole field is a better microphone.
12 concepts in this field
Speech Recognition
Turning spoken audio into text — solved for clear speech in quiet rooms, and still genuinely hard for everything real.
Text-to-Speech
Turning text into speech that sounds human — where the remaining gap isn't the voice, it's knowing which word to stress.
Voice Cloning
Copying a specific person's voice from a short sample — technically impressive, ethically unresolved, and already being used against people.
Speaker Diarization
Working out who spoke when — the unglamorous half of transcription, and usually the half that's wrong.
Music Generation
Models that produce music from a description — good enough for background, and sitting on an unresolved argument about whose work it learned from.
Spectrogram
Turning sound into a picture so a vision model can look at it — the representation nearly all audio AI runs on, and it throws half the signal away.
Vocoder
Turning a spectrogram back into sound — the step that made synthetic speech stop sounding synthetic, and quality was never the bottleneck.
Source Separation
Pulling one voice out of many — the cocktail party problem, named in 1953, and the solution reversed a decades-old assumption about how to represent audio.
Audio Classification
Naming what a sound is — where the field borrowed vision's entire playbook, including its label problems.
Wake Word Detection
Listening for one phrase, always, on a budget of milliwatts — where the privacy guarantee is an engineering constraint rather than a promise.
Voice Conversion
Changing who a recording sounds like while keeping what was said — useful, and the same technology as the fraud.
Speech Emotion Recognition
Detecting how someone feels from their voice — deployed at scale in call centres, and the psychology says the thing it measures may not exist.