Continual Learning
Training a model on new information without destroying what it already knew — the problem catastrophic forgetting names, approached from the side of trying to solve it.
Reviewed July 17, 2026Stable
When not to use it
- Where retraining from scratch is affordable, which is simpler and more predictable.
- Where the task sequence is short and known in advance, which is multi-task training rather than continual learning.
Reach for something else instead
- Periodic full retraining, the standard answer and the reason most models update on a schedule.
- Retrieval, which puts new information outside the weights entirely and avoids the problem.
- Adapter layers, isolating new capability in added parameters rather than modifying existing ones.
Sources & further reading
- Kirkpatrick et al. (2017), Overcoming catastrophic forgetting in neural networks — elastic weight consolidation.
- Parisi et al. (2019), Continual lifelong learning with neural networks: A review — the three families and the stability-plasticity framing.
- De Lange et al. (2021), A continual learning survey — comparative evaluation across methods.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Measuring only the new task. The question is performance across everything seen so far.
- Assuming a large model is immune, when the evidence is unsettled and degradation may be diffuse.
- Fine-tuning narrowly without a held-out general capability set, which is how a targeted improvement quietly costs general competence.
At a glance
FieldMachine Learning
Names the solution tocatastrophic forgetting
Three familiesreplay, regularisation, parameter isolation
Core tensionstability versus plasticity
Statussubstantially unsolved
DifficultyTechnical
Where this sits
A destination. 6 concepts lead here, and nothing in the corpus depends on it.
6Levelsteps in
6Needs firstconcepts
0Opens upnothing further
1Areastays here
Learn these firstCatastrophic Forgetting
Computed from the prerequisite graph, not assigned. How this works