Boosting variational inference: An optimization perspective

conference paper Boosting variational inference: An optimization perspective PMLR published yes Francesco Locatello author 26cfd52f-2483-11ee-8040-88983bcc06d40000-0002-4850-0683 Rajiv Khanna author Joydeep Ghosh author Gunnar Rätsch author FrLo department AISTATS: Conference on Artificial Intelligence and Statistics Variational inference is a popular technique to approximate a possibly intractable Bayesian posterior with a more tractable one. Recently, boosting variational inference has been proposed as a new paradigm to approximate the posterior by a mixture of densities by greedily adding components to the mixture. However, as is the case with many other variational inference algorithms, its theoretical properties have not been studied. In the present work, we study the convergence properties of this approach from a modern optimization viewpoint by establishing connections to the classic Frank-Wolfe algorithm. Our analyses yields novel theoretical insights regarding the sufficient conditions for convergence, explicit rates, and algorithmic simplifications. Since a lot of focus in previous works for variational inference has been on tractability, our work is especially important as a much needed attempt to bridge the gap between probabilistic models and their corresponding theoretical properties. ML Research Press2018Playa Blanca, Lanzarote eng Proceedings of the 21st International Conference on Artificial Intelligence and Statistics 1708.01733 84464-472 yes F. Locatello, R. Khanna, J. Ghosh, and G. Rätsch, “Boosting variational inference: An optimization perspective,” in Proceedings of the 21st International Conference on Artificial Intelligence and Statistics, Playa Blanca, Lanzarote, 2018, vol. 84, pp. 464–472. Locatello, Francesco, Rajiv Khanna, Joydeep Ghosh, and Gunnar Rätsch. “Boosting Variational Inference: An Optimization Perspective.” In Proceedings of the 21st International Conference on Artificial Intelligence and Statistics, 84:464–72. ML Research Press, 2018. Locatello, Francesco, et al. “Boosting Variational Inference: An Optimization Perspective.” Proceedings of the 21st International Conference on Artificial Intelligence and Statistics, vol. 84, ML Research Press, 2018, pp. 464–72. Locatello F, Khanna R, Ghosh J, Rätsch G. 2018. Boosting variational inference: An optimization perspective. Proceedings of the 21st International Conference on Artificial Intelligence and Statistics. AISTATS: Conference on Artificial Intelligence and Statistics, PMLR, vol. 84, 464–472. Locatello F, Khanna R, Ghosh J, Rätsch G. Boosting variational inference: An optimization perspective. In: Proceedings of the 21st International Conference on Artificial Intelligence and Statistics. Vol 84. ML Research Press; 2018:464-472. F. Locatello, R. Khanna, J. Ghosh, G. Rätsch, in:, Proceedings of the 21st International Conference on Artificial Intelligence and Statistics, ML Research Press, 2018, pp. 464–472. Locatello, F., Khanna, R., Ghosh, J., & Rätsch, G. (2018). Boosting variational inference: An optimization perspective. In Proceedings of the 21st International Conference on Artificial Intelligence and Statistics (Vol. 84, pp. 464–472). Playa Blanca, Lanzarote: ML Research Press. 142012023-08-22T14:15:20Z2024-10-14T12:29:27Z