Catastrophic Forgetting
A network learning something new and losing what it already knew — a problem identified in 1989 that fine-tuning made everybody's problem again.
Reviewed July 16, 2026Stable
When not to use it
- As an explanation for a model that was never good at the task. Forgetting means losing something it had; verify it had it.
- Where you're training once. This is a sequential-learning problem; single-task training from scratch doesn't have it.
- As a reason to avoid fine-tuning. It's a reason to measure broadly and prefer parameter-efficient methods, not to skip the tool.
Reach for something else instead
- LoRA and parameter-efficient tuning avoid the problem structurally by leaving base weights untouched.
- RAG adds knowledge with no training at all, and nothing can be forgotten.
- Prompting handles a surprising share of what people reach for fine-tuning to fix, and forgets nothing.
Sources & further reading
- McCloskey & Cohen (1989), Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem — the original identification and mechanism.
- Kirkpatrick et al. (2017), Overcoming Catastrophic Forgetting in Neural Networks — PNAS; Elastic Weight Consolidation.
- Luo et al. (2023), An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning — the modern confirmation, and the finding that it worsens with scale.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Evaluating only the target task. Forgetting is invisible unless you measure capabilities you weren't trying to change — which is exactly what nobody holds out.
- Assuming a bigger model is safer. The empirical finding is the opposite: forgetting gets worse with scale.
- Fine-tuning to add facts. Facts belong in retrieval; fine-tuning shapes behaviour, and paying for it in forgotten capability is a bad trade.
At a glance
FieldDeep Learning
IdentifiedMcCloskey & Cohen, 1989
Causeshared weights, no term protecting the old task
Worsens withmodel scale
DifficultyIntermediate