Traditionally machine learning has been focusing on the problem of solving a single task in isolation. While being quite well understood, this approach disregards an important aspect of human learning: when facing a new problem, humans are able to exploit knowledge acquired from previously learned tasks. Intuitively, access to several problems simultaneously or sequentially could also be advantageous for a machine learning system, especially if these tasks are closely related. Indeed, results of many empirical studies have provided justification for this intuition. However, theoretical justifications of this idea are rather limited. The focus of this thesis is to expand the understanding of potential benefits of information transfer between several related learning problems. We provide theoretical analysis for three scenarios of multi-task learning - multiple kernel learning, sequential learning and active task selection. We also provide a PAC-Bayesian perspective on lifelong learning and investigate how the task generation process influences the generalization guarantees in this scenario. In addition, we show how some of the obtained theoretical results can be used to derive principled multi-task and lifelong learning algorithms and illustrate their performance on various synthetic and real-world datasets.
First and foremost I would like to express my gratitude to my supervisor, Christoph Lampert. Thank you for your patience in teaching me all aspects of doing research (including English grammar), for your trust in my capabilities and endless support. Thank you for granting me freedom in my research and, at the same time, having time and helping me cope with the consequences whenever I needed it. Thank you for creating an excellent atmosphere in the group, it was a great pleasure and honor to be a part of it. There could not have been a better and more inspiring adviser and mentor. I thank Shai Ben-David for welcoming me into his group at the University of Waterloo, for inspiring discussions and support. It was a great pleasure to work together. I am also thankful to Ruth Urner for hosting me at the Max-Planck Institute Tübingen, for the fruitful collaboration and for taking care of me during that not-so-sunny month of May. I thank Jan Maas for kindly joining my thesis committee despite the short notice and providing me with insightful comments. I would like to thank my colleagues for their support, entertaining conversations and endless table soccer games we shared together: Georg, Jan, Amelie and Emilie, Michal and Alex, Alex K. and Alex Z., Thomas, Sameh, Vlad, Mayu, Nathaniel, Silvester, Neel, Csaba, Vladimir, Morten. Thank you, Mabel and Ram, for the wonderful time we spent together. I am thankful to Shrinu and Samira for taking care of me during my stay at the University of Waterloo. Special thanks to Viktoriia for her never-ending optimism and for being so inspiring and supportive, especially at the beginning of my PhD journey. Thanks to IST administration, in particular, Vlad and Elisabeth for shielding me from most of the bureaucratic paperwork. This dissertation would not have been possible without funding from the European Research Council under the European Union's Seventh Framework Programme (FP7/2007-2013)/ERC grant agreement no 308036.
Pentina A. Theoretical foundations of multi-task lifelong learning. 2016. doi:10.15479/AT:ISTA:TH_776
Pentina, A. (2016). Theoretical foundations of multi-task lifelong learning. Institute of Science and Technology Austria. https://doi.org/10.15479/AT:ISTA:TH_776
Pentina, Anastasia. “Theoretical Foundations of Multi-Task Lifelong Learning.” Institute of Science and Technology Austria, 2016. https://doi.org/10.15479/AT:ISTA:TH_776.
A. Pentina, “Theoretical foundations of multi-task lifelong learning,” Institute of Science and Technology Austria, 2016.
Pentina A. 2016. Theoretical foundations of multi-task lifelong learning. Institute of Science and Technology Austria.
Pentina, Anastasia. Theoretical Foundations of Multi-Task Lifelong Learning. Institute of Science and Technology Austria, 2016, doi:10.15479/AT:ISTA:TH_776.