Theoretical foundations of multi-task lifelong learning
Pentina A. 2016. Theoretical foundations of multi-task lifelong learning. Institute of Science and Technology Austria.
Download
Thesis
| PhD
| Published
| English
Author
Supervisor
Corresponding author has ISTA affiliation
Department
Series Title
ISTA Thesis
Abstract
Traditionally machine learning has been focusing on the problem of solving a single
task in isolation. While being quite well understood, this approach disregards an
important aspect of human learning: when facing a new problem, humans are able to
exploit knowledge acquired from previously learned tasks. Intuitively, access to several
problems simultaneously or sequentially could also be advantageous for a machine
learning system, especially if these tasks are closely related. Indeed, results of many
empirical studies have provided justification for this intuition. However, theoretical
justifications of this idea are rather limited.
The focus of this thesis is to expand the understanding of potential benefits of information
transfer between several related learning problems. We provide theoretical
analysis for three scenarios of multi-task learning - multiple kernel learning, sequential
learning and active task selection. We also provide a PAC-Bayesian perspective on
lifelong learning and investigate how the task generation process influences the generalization
guarantees in this scenario. In addition, we show how some of the obtained
theoretical results can be used to derive principled multi-task and lifelong learning
algorithms and illustrate their performance on various synthetic and real-world datasets.
Publishing Year
Date Published
2016-11-01
Publisher
Institute of Science and Technology Austria
Acknowledgement
First and foremost I would like to express my gratitude to my supervisor, Christoph
Lampert. Thank you for your patience in teaching me all aspects of doing research
(including English grammar), for your trust in my capabilities and endless support. Thank
you for granting me freedom in my research and, at the same time, having time and
helping me cope with the consequences whenever I needed it. Thank you for creating
an excellent atmosphere in the group, it was a great pleasure and honor to be a part of
it. There could not have been a better and more inspiring adviser and mentor.
I thank Shai Ben-David for welcoming me into his group at the University of Waterloo,
for inspiring discussions and support. It was a great pleasure to work together. I am
also thankful to Ruth Urner for hosting me at the Max-Planck Institute Tübingen, for the
fruitful collaboration and for taking care of me during that not-so-sunny month of May.
I thank Jan Maas for kindly joining my thesis committee despite the short notice and
providing me with insightful comments.
I would like to thank my colleagues for their support, entertaining conversations and
endless table soccer games we shared together: Georg, Jan, Amelie and Emilie, Michal
and Alex, Alex K. and Alex Z., Thomas, Sameh, Vlad, Mayu, Nathaniel, Silvester, Neel,
Csaba, Vladimir, Morten. Thank you, Mabel and Ram, for the wonderful time we spent
together. I am thankful to Shrinu and Samira for taking care of me during my stay at the
University of Waterloo. Special thanks to Viktoriia for her never-ending optimism and for
being so inspiring and supportive, especially at the beginning of my PhD journey.
Thanks to IST administration, in particular, Vlad and Elisabeth for shielding me from
most of the bureaucratic paperwork.
This dissertation would not have been possible without funding from the European
Research Council under the European Union's Seventh Framework Programme
(FP7/2007-2013)/ERC grant agreement no 308036.
Page
127
ISSN
IST-REx-ID
Cite this
Pentina A. Theoretical foundations of multi-task lifelong learning. 2016. doi:10.15479/AT:ISTA:TH_776
Pentina, A. (2016). Theoretical foundations of multi-task lifelong learning. Institute of Science and Technology Austria. https://doi.org/10.15479/AT:ISTA:TH_776
Pentina, Anastasia. “Theoretical Foundations of Multi-Task Lifelong Learning.” Institute of Science and Technology Austria, 2016. https://doi.org/10.15479/AT:ISTA:TH_776.
A. Pentina, “Theoretical foundations of multi-task lifelong learning,” Institute of Science and Technology Austria, 2016.
Pentina A. 2016. Theoretical foundations of multi-task lifelong learning. Institute of Science and Technology Austria.
Pentina, Anastasia. Theoretical Foundations of Multi-Task Lifelong Learning. Institute of Science and Technology Austria, 2016, doi:10.15479/AT:ISTA:TH_776.
All files available under the following license(s):
Copyright Statement:
This Item is protected by copyright and/or related rights. [...]
Main File(s)
File Name
Access Level
Open Access
Date Uploaded
2018-12-12