Catastrophic Forgetting
A network learning something new and losing what it already knew — a problem identified in 1989 that fine-tuning made everybody's problem again.
Reviewed July 14, 2026Stable
When not to use it
- As an explanation for a model that was never good at the task. Forgetting means losing something it had; verify it had it.
- Where you're training once. This is a sequential-learning problem; single-task training from scratch doesn't have it.
- As a reason to avoid fine-tuning. It's a reason to measure broadly and prefer parameter-efficient methods, not to skip the tool.
Reach for something else instead
- LoRA and parameter-efficient tuning avoid the problem structurally by leaving base weights untouched.
- RAG adds knowledge with no training at all, and nothing can be forgotten.
- Prompting handles a surprising share of what people reach for fine-tuning to fix, and forgets nothing.
Read more on the blog
- Why AI models get worse: forgetting, collapse, and driftThree different mechanisms quietly degrade AI systems, catastrophic forgetting, model collapse, and data drift. They get blamed for each other constantly. Here's how to tell them apart, and which one is actually eating your accuracy.
- What is fine-tuning? How to specialize an AI modelFine-tuning takes a general-purpose model and adapts it to your specific task by continuing its training on your examples. It is one of the three ways to shape a model, the most powerful and the most misused. Here is what it actually does, how LoRA made it cheap, and the honest answer to whether you should do it.
- How AI models are trained: from raw text to a system that helpsA language model isn't programmed, it's grown, in stages, from a firehose of text into a system that answers helpfully. The full modern pipeline: pretraining, supervised fine-tuning, preference alignment, and the reasoning training that defines 2026, what each stage does, why none can be skipped, and how the recipe changed.
- RAG vs fine-tuning: the decision, honestlyThe most common question in applied AI, answered without the hedging: when do you use retrieval, when do you fine-tune, when do you need both, and what almost every team gets wrong about the choice.
Further reading
- McCloskey & Cohen (1989), Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem — the original identification and mechanism.
- Kirkpatrick et al. (2017), Overcoming Catastrophic Forgetting in Neural Networks — PNAS; Elastic Weight Consolidation.
- Luo et al. (2023), An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning — the modern confirmation, and the finding that it worsens with scale.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Evaluating only the target task. Forgetting is invisible unless you measure capabilities you weren't trying to change — which is exactly what nobody holds out.
- Assuming a bigger model is safer. The empirical finding is the opposite: forgetting gets worse with scale.
- Fine-tuning to add facts. Facts belong in retrieval; fine-tuning shapes behaviour, and paying for it in forgotten capability is a bad trade.
At a glance
FieldDeep Learning
IdentifiedMcCloskey & Cohen, 1989
Causeshared weights, no term protecting the old task
Worsens withmodel scale
DifficultyIntermediate
Where this sits
5 concepts come first. Understanding it opens up 1 more.
5Levelsteps in
5Needs firstconcepts
1Opens up0% of 310
2Areascrosses fields
Learn these firstTransfer Learning
Computed from the prerequisite graph, not assigned. How this works