Encoder-Decoder
Read the whole input into a representation, then generate the output from it — the architecture that made translation work and that attention was invented to fix.
Reviewed July 16, 2026Contested
When not to use it
- For open-ended chat and general instruction following, where decoder-only models are simpler and stronger.
- For pure classification, where an encoder alone is sufficient and cheaper.
Reach for something else instead
- Decoder-only for generation and general-purpose use.
- Encoder-only for embeddings, retrieval and classification.
- Prefix language models, which blur the boundary by allowing bidirectional attention over a prompt.
Sources & further reading
- Sutskever, Vinyals & Le (2014), Sequence to Sequence Learning with Neural Networks — the original encoder-decoder result.
- Bahdanau et al. (2015), Neural Machine Translation by Jointly Learning to Align and Translate — attention introduced to fix the bottleneck.
- Raffel et al. (2020), Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer — the T5 case for encoder-decoder at scale.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Assuming attention originated with transformers. It was introduced to fix the encoder-decoder bottleneck two years earlier.
- Using an encoder-decoder for chat, where the input and output are not distinct objects.
- Treating decoder-only dominance as settled evidence of architectural superiority rather than of where scale was applied.
At a glance
FieldDeep Learning
Splits intoencoder plus decoder
Originatedneural machine translation
Bottleneckfixed-size representation, fixed by attention
Best fortranslation and speech
DifficultyHands-on
Where this sits
5 concepts come first. Understanding it opens up 71 more.
5Levelsteps in
5Needs firstconcepts
71Opens up27% of 264
8Areascrosses fields
Learn these firstRNN
Computed from the prerequisite graph, not assigned. How this works