---
_id: '11452'
abstract:
- lang: eng
  text: We study efficient distributed algorithms for the fundamental problem of principal
    component analysis and leading eigenvector computation on the sphere, when the
    data are randomly distributed among a set of computational nodes. We propose a
    new quantized variant of Riemannian gradient descent to solve this problem, and
    prove that the algorithm converges with high probability under a set of necessary
    spherical-convexity properties. We give bounds on the number of bits transmitted
    by the algorithm under common initialization schemes, and investigate the dependency
    on the problem dimension in each case.
acknowledgement: We would like to thank the anonymous reviewers for helpful comments
  and suggestions. We also thank Aurelien Lucchi and Antonio Orvieto for fruitful
  discussions at an early stage of this work. FA is partially supported by the SNSF
  under research project No. 192363 and conducted part of this work while at IST Austria
  under the European Union’s Horizon 2020 research and innovation programme (grant
  agreement No. 805223 ScaleML). PD partly conducted this work while at IST Austria
  and was supported by the European Union’s Horizon 2020 programme under the Marie
  Skłodowska-Curie grant agreement No. 754411.
article_processing_charge: No
arxiv: 1
author:
- first_name: Foivos
  full_name: Alimisis, Foivos
  last_name: Alimisis
- first_name: Peter
  full_name: Davies, Peter
  id: 11396234-BB50-11E9-B24C-90FCE5697425
  last_name: Davies
  orcid: 0000-0002-5646-9524
- first_name: Bart
  full_name: Vandereycken, Bart
  last_name: Vandereycken
- first_name: Dan-Adrian
  full_name: Alistarh, Dan-Adrian
  id: 4A899BFC-F248-11E8-B48F-1D18A9856A87
  last_name: Alistarh
  orcid: 0000-0003-3650-940X
citation:
  ama: 'Alimisis F, Davies P, Vandereycken B, Alistarh D-A. Distributed principal
    component analysis with limited communication. In: <i>Advances in Neural Information
    Processing Systems - 35th Conference on Neural Information Processing Systems</i>.
    Vol 4. Neural Information Processing Systems Foundation; 2021:2823-2834.'
  apa: 'Alimisis, F., Davies, P., Vandereycken, B., &#38; Alistarh, D.-A. (2021).
    Distributed principal component analysis with limited communication. In <i>Advances
    in Neural Information Processing Systems - 35th Conference on Neural Information
    Processing Systems</i> (Vol. 4, pp. 2823–2834). Virtual, Online: Neural Information
    Processing Systems Foundation.'
  chicago: Alimisis, Foivos, Peter Davies, Bart Vandereycken, and Dan-Adrian Alistarh.
    “Distributed Principal Component Analysis with Limited Communication.” In <i>Advances
    in Neural Information Processing Systems - 35th Conference on Neural Information
    Processing Systems</i>, 4:2823–34. Neural Information Processing Systems Foundation,
    2021.
  ieee: F. Alimisis, P. Davies, B. Vandereycken, and D.-A. Alistarh, “Distributed
    principal component analysis with limited communication,” in <i>Advances in Neural
    Information Processing Systems - 35th Conference on Neural Information Processing
    Systems</i>, Virtual, Online, 2021, vol. 4, pp. 2823–2834.
  ista: 'Alimisis F, Davies P, Vandereycken B, Alistarh D-A. 2021. Distributed principal
    component analysis with limited communication. Advances in Neural Information
    Processing Systems - 35th Conference on Neural Information Processing Systems.
    NeurIPS: Neural Information Processing Systems vol. 4, 2823–2834.'
  mla: Alimisis, Foivos, et al. “Distributed Principal Component Analysis with Limited
    Communication.” <i>Advances in Neural Information Processing Systems - 35th Conference
    on Neural Information Processing Systems</i>, vol. 4, Neural Information Processing
    Systems Foundation, 2021, pp. 2823–34.
  short: F. Alimisis, P. Davies, B. Vandereycken, D.-A. Alistarh, in:, Advances in
    Neural Information Processing Systems - 35th Conference on Neural Information
    Processing Systems, Neural Information Processing Systems Foundation, 2021, pp.
    2823–2834.
conference:
  end_date: 2021-12-14
  location: Virtual, Online
  name: 'NeurIPS: Neural Information Processing Systems'
  start_date: 2021-12-06
corr_author: '1'
date_created: 2022-06-19T22:01:58Z
date_published: 2021-12-01T00:00:00Z
date_updated: 2025-04-14T07:43:57Z
day: '01'
department:
- _id: DaAl
ec_funded: 1
external_id:
  arxiv:
  - '2110.14391'
intvolume: '         4'
language:
- iso: eng
main_file_link:
- open_access: '1'
  url: https://proceedings.neurips.cc/paper/2021/file/1680e9fa7b4dd5d62ece800239bb53bd-Paper.pdf
month: '12'
oa: 1
oa_version: Published Version
page: 2823-2834
project:
- _id: 268A44D6-B435-11E9-9278-68D0E5697425
  call_identifier: H2020
  grant_number: '805223'
  name: Elastic Coordination for Scalable Machine Learning
- _id: 260C2330-B435-11E9-9278-68D0E5697425
  call_identifier: H2020
  grant_number: '754411'
  name: ISTplus - Postdoctoral Fellowships
publication: Advances in Neural Information Processing Systems - 35th Conference on
  Neural Information Processing Systems
publication_identifier:
  isbn:
  - '9781713845393'
  issn:
  - 1049-5258
publication_status: published
publisher: Neural Information Processing Systems Foundation
quality_controlled: '1'
scopus_import: '1'
status: public
title: Distributed principal component analysis with limited communication
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 4
year: '2021'
...