DeepMind researchers detail algorithm for training neural nets sequentially at multiple tasks, allowing neural nets to retain knowledge while learning new tasks
Breakthrough should improve ability to learn sequence of tasks — Research may also have implication for study of brain synapses
Context & Ripple Effects
DeepMind's sequential-training result extends a line of research into making neural networks learn more like animals do: three years earlier, researchers had shown an algorithm teaching a computer a new concept from a single example, and this work attacks the complementary problem — holding onto old skills while acquiring new ones.
The arc has since run long: OpenAI's Jeff Clune later presented ANML, which learns 600 sequential tasks with minimal catastrophic forgetting, and by late 2025 Bloomberg reported AI leaders explicitly pursuing continual learning as a route to more powerful systems — making this 2017 paper an early data point in what became a named research agenda.
First-order effects
- DeepMind gains a training procedure that lets a single network absorb a sequence of tasks without erasing prior knowledge, removing the need to retrain from scratch per task.
- The researchers also open a bridge to neuroscience, with the mechanism offering a possible model for how brain synapses preserve memory during learning.
Second-order effects
- Rival labs are pushed to compete on retention as well as raw capability — OpenAI's subsequent ANML result frames 'how many sequential tasks with minimal forgetting' as a benchmark to beat.
- If sequential training matures, compute spent retraining models for each new task can be redirected toward extending existing ones, shifting where training budgets go.
Third-order effects
- A durable fix for catastrophic forgetting points toward continuously updated models rather than frozen snapshots — the endpoint the 2025 coverage describes as continual learning.
- It also reframes the 2020 finding that efficiency-focused algorithms had barely improved over a decade: progress may come less from squeezing networks than from changing how they accumulate knowledge over time.
The trend: Continual learning is moving from a niche robustness problem toward a core strategy for building more capable AI systems, with each retention advance resetting what labs consider trainable.