Vocoder
Turning a spectrogram back into sound — the step that made synthetic speech stop sounding synthetic, and quality was never the bottleneck.
When not to use it
- Griffin-Lim, for anything you'll ship. It guesses phase and sounds like it.
- Autoregressive vocoders in production. 16,000 sequential passes per second of audio.
- A separate vocoder, if end-to-end fits. Not creating the phase problem beats solving it.
- Diffusion vocoders where latency matters. Great quality, iterative, back to the speed problem.
Reach for something else instead
- HiFi-GAN — fast, excellent, what you'd use.
- End-to-end TTS (VITS) — no intermediate spectrogram, no phase to invent.
- Neural codecs — audio as tokens; the direction everything is moving.
- Concatenative synthesis — splice real recordings. Ancient, and it never had a phase problem.
Sources & further reading
- van den Oord et al. (2016), WaveNet: A Generative Model for Raw Audio — quality solved, speed impossible. Dilated causal convolutions outlived it.
- Kong, Kim & Bae (2020), HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis — what won; period-based discriminators.
- Défossez et al. (2022), High Fidelity Neural Audio Compression — EnCodec; audio as discrete tokens, which is the bigger shift.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Thinking quality was the bottleneck. WaveNet solved quality in 2016; seven years went into speed.
- Using Griffin-Lim and blaming the acoustic model. The metallic sound is the phase guess.
- Missing why GANs suit this — the discriminator learns coherent phase with no explicit loss for it.
- Treating dilated convolutions as a WaveNet detail. Exponential receptive field per layer outlived the model.
At a glance
FieldSpeech & Audio
The jobspectrogram back to waveform; invent the phase that was thrown away
WaveNet (2016)human quality, 16,000 sequential passes per second of audio
What wonHiFi-GAN; period-based discriminators, real-time
The real storyquality was solved first; everything since was speed
DifficultyAdvanced
Flashcards for this concept
Question
—
Answer
—
1 / 4
Often compared with
Vocoder vs. end-to-end TTS — one inverts a spectrogram and has to invent the phase; the other never made a spectrogram, so there's no phase to invent.