Speech Recognition
Turning spoken audio into text — solved for clear speech in quiet rooms, and still genuinely hard for everything real.
When not to use it
- When the audio is bad and you can fix the audio. A better microphone beats a better model. Spend there first.
- When errors are expensive and unreviewed. At 5% WER, one word in twenty is wrong. If that word is a dosage or a name, you need a human in the loop.
- On heavily overlapping speech, without diarization. Two people at once is a different problem, and transcription alone will blend them into nonsense.
- When a structured input would do. If you need a date and a number, a form is more reliable than transcribing someone saying them.
Reach for something else instead
- Human transcription — still better on hard audio, and the honest choice for high-stakes content.
- Constrained voice input — a limited grammar ("say yes or no") is far more reliable than open recognition.
- Keyword spotting — if you only need to detect a few phrases, you don't need full transcription.
- Fine-tuned domain models — when vocabulary is the bottleneck and you have labelled audio.
Sources & further reading
- Radford et al. (2022), Robust Speech Recognition via Large-Scale Weak Supervision — Whisper; robustness bought with data rather than architecture.
- Graves et al. (2006), Connectionist Temporal Classification — the alignment method that made end-to-end speech recognition possible.
- Koenecke et al. (2020), Racial Disparities in Automated Speech Recognition — measured performance gaps across speaker groups; the fairness problem stated with numbers.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Judging a system on benchmark WER rather than on your own audio. Your acoustic conditions are the variable that matters.
- Not using vocabulary biasing. It's free, it's the biggest available win on domain terms, and most people don't know it's there.
- Assuming silence produces silence. Attention-based models can hallucinate fluent sentences from noise.
- Ignoring accent disparity because the average number looks fine. The average hides the group being failed.
- Choosing streaming when batch would do. You're paying accuracy for latency you may not need.
At a glance
FieldSpeech & Audio
Also calledASR, speech-to-text
Metricword error rate
Biggest leveraudio quality
Second biggestvocabulary biasing
Still hardoverlap, accents, code-switching
DifficultyBeginner
Flashcards for this concept
Question
—
Answer
—
1 / 4
Often compared with
Streaming vs. batch recognition — words as you speak vs. words when you finish. Streaming can't use future context to fix earlier guesses, so it's less accurate by construction.