Distributed principal component analysis with limited communication

conference paper Distributed principal component analysis with limited communication published yes Foivos Alimisis author Peter Davies author 11396234-BB50-11E9-B24C-90FCE56974250000-0002-5646-9524 Bart Vandereycken author Dan-Adrian Alistarh author 4A899BFC-F248-11E8-B48F-1D18A9856A870000-0003-3650-940X DaAl department NeurIPS: Neural Information Processing Systems Elastic Coordination for Scalable Machine Learning project ISTplus - Postdoctoral Fellowships project We study efficient distributed algorithms for the fundamental problem of principal component analysis and leading eigenvector computation on the sphere, when the data are randomly distributed among a set of computational nodes. We propose a new quantized variant of Riemannian gradient descent to solve this problem, and prove that the algorithm converges with high probability under a set of necessary spherical-convexity properties. We give bounds on the number of bits transmitted by the algorithm under common initialization schemes, and investigate the dependency on the problem dimension in each case. Neural Information Processing Systems Foundation2021Virtual, Online eng Advances in Neural Information Processing Systems - 35th Conference on Neural Information Processing Systems 1049-5258 9781713845393 2110.14391 42823-2834 Alimisis F, Davies P, Vandereycken B, Alistarh D-A. 2021. Distributed principal component analysis with limited communication. Advances in Neural Information Processing Systems - 35th Conference on Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems vol. 4, 2823–2834. F. Alimisis, P. Davies, B. Vandereycken, D.-A. Alistarh, in:, Advances in Neural Information Processing Systems - 35th Conference on Neural Information Processing Systems, Neural Information Processing Systems Foundation, 2021, pp. 2823–2834. Alimisis, F., Davies, P., Vandereycken, B., & Alistarh, D.-A. (2021). Distributed principal component analysis with limited communication. In Advances in Neural Information Processing Systems - 35th Conference on Neural Information Processing Systems (Vol. 4, pp. 2823–2834). Virtual, Online: Neural Information Processing Systems Foundation. Alimisis, Foivos, et al. “Distributed Principal Component Analysis with Limited Communication.” Advances in Neural Information Processing Systems - 35th Conference on Neural Information Processing Systems, vol. 4, Neural Information Processing Systems Foundation, 2021, pp. 2823–34. F. Alimisis, P. Davies, B. Vandereycken, and D.-A. Alistarh, “Distributed principal component analysis with limited communication,” in Advances in Neural Information Processing Systems - 35th Conference on Neural Information Processing Systems, Virtual, Online, 2021, vol. 4, pp. 2823–2834. Alimisis, Foivos, Peter Davies, Bart Vandereycken, and Dan-Adrian Alistarh. “Distributed Principal Component Analysis with Limited Communication.” In Advances in Neural Information Processing Systems - 35th Conference on Neural Information Processing Systems, 4:2823–34. Neural Information Processing Systems Foundation, 2021. Alimisis F, Davies P, Vandereycken B, Alistarh D-A. Distributed principal component analysis with limited communication. In: Advances in Neural Information Processing Systems - 35th Conference on Neural Information Processing Systems. Vol 4. Neural Information Processing Systems Foundation; 2021:2823-2834. 114522022-06-19T22:01:58Z2025-04-14T07:43:57Z