Continual learning: Applications and the road forward
Verwimp E, Aljundi R, Ben-David S, Bethge M, Cossu A, Gepperth A, Hayes TL, Hüllermeier E, Kanan C, Kudithipudi D, Lampert C, Mundt M, Pascanu R, Popescu A, Tolias AS, Van De Weijer J, Liu B, Lomonaco V, Tuytelaars T, Van De Ven GM. 2024. Continual learning: Applications and the road forward. Transactions on Machine Learning Research. 2024.
Download
Journal Article
| Published
| English
Scopus indexed
Author
Verwimp, Eli;
Aljundi, Rahaf;
Ben-David, Shai;
Bethge, Matthias;
Cossu, Andrea;
Gepperth, Alexander;
Hayes, Tyler L.;
Hüllermeier, Eyke;
Kanan, Christopher;
Kudithipudi, Dhireesha;
Lampert , ChristophISTA
;
Mundt, Martin
All

All
Department
Series Title
TMLR
Abstract
Continual learning is a subfield of machine learning, which aims to allow machine learning models to continuously learn on new data, by accumulating knowledge without forgetting what was learned in the past. In this work, we take a step back, and ask: "Why should one care about continual learning in the first place?". We set the stage by examining recent continual learning papers published at four major machine learning conferences, and show that memory-constrained settings dominate the field. Then, we discuss five open problems in machine learning, and even though they might seem unrelated to continual learning at first sight, we show that continual learning will inevitably be part of their solution. These problems are model editing, personalization and specialization, on-device learning, faster (re-)training and reinforcement learning. Finally, by comparing the desiderata from these unsolved problems and the current assumptions in continual learning, we highlight and discuss four future directions for continual learning research. We hope that this work offers an interesting perspective on the future of continual learning, while displaying its potential value and the paths we have to pursue in order to make it successful. This work is the result of the many discussions the authors had at the Dagstuhl seminar on Deep Continual Learning, in March 2023.
Publishing Year
Date Published
2024-04-12
Journal Title
Transactions on Machine Learning Research
Publisher
Transactions on Machine Learning Research
Volume
2024
eISSN
IST-REx-ID
Cite this
Verwimp E, Aljundi R, Ben-David S, et al. Continual learning: Applications and the road forward. Transactions on Machine Learning Research. 2024;2024.
Verwimp, E., Aljundi, R., Ben-David, S., Bethge, M., Cossu, A., Gepperth, A., … Van De Ven, G. M. (2024). Continual learning: Applications and the road forward. Transactions on Machine Learning Research. Transactions on Machine Learning Research.
Verwimp, Eli, Rahaf Aljundi, Shai Ben-David, Matthias Bethge, Andrea Cossu, Alexander Gepperth, Tyler L. Hayes, et al. “Continual Learning: Applications and the Road Forward.” Transactions on Machine Learning Research. Transactions on Machine Learning Research, 2024.
E. Verwimp et al., “Continual learning: Applications and the road forward,” Transactions on Machine Learning Research, vol. 2024. Transactions on Machine Learning Research, 2024.
Verwimp E, Aljundi R, Ben-David S, Bethge M, Cossu A, Gepperth A, Hayes TL, Hüllermeier E, Kanan C, Kudithipudi D, Lampert C, Mundt M, Pascanu R, Popescu A, Tolias AS, Van De Weijer J, Liu B, Lomonaco V, Tuytelaars T, Van De Ven GM. 2024. Continual learning: Applications and the road forward. Transactions on Machine Learning Research. 2024.
Verwimp, Eli, et al. “Continual Learning: Applications and the Road Forward.” Transactions on Machine Learning Research, vol. 2024, Transactions on Machine Learning Research, 2024.
All files available under the following license(s):
Creative Commons Attribution 4.0 International Public License (CC-BY 4.0):
Main File(s)
File Name
2024_TMLR_Verwimp.pdf
1.37 MB
Access Level

Date Uploaded
2025-03-20
MD5 Checksum
0714e12f7423cd098976ed9974561155
Export
Marked PublicationsOpen Data ISTA Research Explorer
Sources
arXiv 2311.11908