Distributed principal component analysis with limited communication
Alimisis F, Davies P, Vandereycken B, Alistarh D-A. 2021. Distributed principal component analysis with limited communication. Advances in Neural Information Processing Systems - 35th Conference on Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems vol. 4, 2823–2834.
Download (ext.)
Conference Paper
| Published
| English
Scopus indexed
Author
Corresponding author has ISTA affiliation
Department
Abstract
We study efficient distributed algorithms for the fundamental problem of principal component analysis and leading eigenvector computation on the sphere, when the data are randomly distributed among a set of computational nodes. We propose a new quantized variant of Riemannian gradient descent to solve this problem, and prove that the algorithm converges with high probability under a set of necessary spherical-convexity properties. We give bounds on the number of bits transmitted by the algorithm under common initialization schemes, and investigate the dependency on the problem dimension in each case.
Publishing Year
Date Published
2021-12-01
Proceedings Title
Advances in Neural Information Processing Systems - 35th Conference on Neural Information Processing Systems
Publisher
Neural Information Processing Systems Foundation
Acknowledgement
We would like to thank the anonymous reviewers for helpful comments and suggestions. We also thank Aurelien Lucchi and Antonio Orvieto for fruitful discussions at an early stage of this work. FA is partially supported by the SNSF under research project No. 192363 and conducted part of this work while at IST Austria under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 805223 ScaleML). PD partly conducted this work while at IST Austria and was supported by the European Union’s Horizon 2020 programme under the Marie Skłodowska-Curie grant agreement No. 754411.
Volume
4
Page
2823-2834
Conference
NeurIPS: Neural Information Processing Systems
Conference Location
Virtual, Online
Conference Date
2021-12-06 – 2021-12-14
ISBN
ISSN
IST-REx-ID
Cite this
Alimisis F, Davies P, Vandereycken B, Alistarh D-A. Distributed principal component analysis with limited communication. In: Advances in Neural Information Processing Systems - 35th Conference on Neural Information Processing Systems. Vol 4. Neural Information Processing Systems Foundation; 2021:2823-2834.
Alimisis, F., Davies, P., Vandereycken, B., & Alistarh, D.-A. (2021). Distributed principal component analysis with limited communication. In Advances in Neural Information Processing Systems - 35th Conference on Neural Information Processing Systems (Vol. 4, pp. 2823–2834). Virtual, Online: Neural Information Processing Systems Foundation.
Alimisis, Foivos, Peter Davies, Bart Vandereycken, and Dan-Adrian Alistarh. “Distributed Principal Component Analysis with Limited Communication.” In Advances in Neural Information Processing Systems - 35th Conference on Neural Information Processing Systems, 4:2823–34. Neural Information Processing Systems Foundation, 2021.
F. Alimisis, P. Davies, B. Vandereycken, and D.-A. Alistarh, “Distributed principal component analysis with limited communication,” in Advances in Neural Information Processing Systems - 35th Conference on Neural Information Processing Systems, Virtual, Online, 2021, vol. 4, pp. 2823–2834.
Alimisis F, Davies P, Vandereycken B, Alistarh D-A. 2021. Distributed principal component analysis with limited communication. Advances in Neural Information Processing Systems - 35th Conference on Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems vol. 4, 2823–2834.
Alimisis, Foivos, et al. “Distributed Principal Component Analysis with Limited Communication.” Advances in Neural Information Processing Systems - 35th Conference on Neural Information Processing Systems, vol. 4, Neural Information Processing Systems Foundation, 2021, pp. 2823–34.
All files available under the following license(s):
Copyright Statement:
This Item is protected by copyright and/or related rights. [...]
Link(s) to Main File(s)
Access Level
Open Access
Export
Marked PublicationsOpen Data ISTA Research Explorer
Sources
arXiv 2110.14391
Google Scholar
ISBN Search