Voice Activity Detection
Deciding which parts of an audio stream contain speech at all — the first stage of almost every speech system, and the one whose errors nothing downstream can undo.
Reviewed July 17, 2026Stable
When not to use it
- On audio you know is continuous speech, where it can only remove things.
- With aggressive settings on quiet or distant speakers, which is how soft-spoken participants disappear from transcripts.
Reach for something else instead
- End-to-end systems that consume raw audio and learn to ignore non-speech implicitly.
- Per-channel recording, where a channel's activity is already known.
- Energy thresholding for controlled quiet environments, which is nearly free and brittle.
Sources & further reading
- Sohn, Kim & Sung (1999), A statistical model-based voice activity detection — the classical statistical formulation.
- Jia et al. (2021), MarbleNet: Deep 1D Time-Channel Separable Convolutional Neural Network for Voice Activity Detection — a compact modern neural detector.
- Bredin & Laurent (2021), End-to-end speaker segmentation for overlap-aware resegmentation — how detection and segmentation merge in current diarization pipelines.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Treating it as solved plumbing. In noisy audio it can dominate total pipeline error.
- Tuning it on clean samples and deploying on noisy ones, since the operating point does not transfer.
- Reporting downstream accuracy with oracle detection and presenting it as deployment performance.
At a glance
FieldSpeech & Audio
Runsfirst, before everything
Discardstypically 30 to 60% of meeting audio
Failure modesmisses are unrecoverable, false alarms create phantoms
Key settingaggressiveness
DifficultyPractical
Where this sits
1 concept come first. Understanding it opens up 1 more.
1Levelsteps in
1Needs firstconcepts
1Opens up0% of 264
1Areastays here
Learn these firstSpectrogram
Computed from the prerequisite graph, not assigned. How this works