Classical continual learning (CL) has long focused on parameter-centric mechanisms—training strategies, architectural designs, and weight adaptation—to help models update and retain knowledge. However, a new research paper from Hugging Face argues that emerging paradigms are broadening the scope of CL beyond this traditional view.
The paper highlights three key developments: on-policy learning expands the space of update mechanisms; test-time training extends CL from the training phase to inference; and external harness components such as memory, skill libraries, and interaction protocols push the evolutionary boundaries of model capabilities far beyond static parameters. Together, these signal a shift from parameter-centric learning toward system-level adaptation.
To characterize this transition, the authors propose a tri-axial framework examining When, How, and Where learning occurs. The How dimension covers off-policy, on-policy, and beyond-gradient optimization mechanics; the When dimension captures evolution across pre-training, post-training, and inference-time stages; and the Where dimension delineates updates within internal parameters versus external structural constraints.
Using this framework, the paper systematically surveys representative methods, traces the ongoing transition of continual learning, and discusses key challenges, broader implications, and future directions arising from this paradigm shift.