Fundamental limits in structured principal component analysis and how to reach them
Barbier J, Camilli F, Mondelli M, Sáenz M. 2023. Fundamental limits in structured principal component analysis and how to reach them. Proceedings of the National Academy of Sciences of the United States of America. 120(30), e2302028120.
Download
Journal Article
| Published
| English
Scopus indexed
Author
Barbier, Jean;
Camilli, Francesco;
Mondelli, MarcoISTA ;
Sáenz, Manuel
Department
Abstract
How do statistical dependencies in measurement noise influence high-dimensional inference? To answer this, we study the paradigmatic spiked matrix model of principal components analysis (PCA), where a rank-one matrix is corrupted by additive noise. We go beyond the usual independence assumption on the noise entries, by drawing the noise from a low-order polynomial orthogonal matrix ensemble. The resulting noise correlations make the setting relevant for applications but analytically challenging. We provide characterization of the Bayes optimal limits of inference in this model. If the spike is rotation invariant, we show that standard spectral PCA is optimal. However, for more general priors, both PCA and the existing approximate message-passing algorithm (AMP) fall short of achieving the information-theoretic limits, which we compute using the replica method from statistical physics. We thus propose an AMP, inspired by the theory of adaptive Thouless–Anderson–Palmer equations, which is empirically observed to saturate the conjectured theoretical limit. This AMP comes with a rigorous state evolution analysis tracking its performance. Although we focus on specific noise distributions, our methodology can be generalized to a wide class of trace matrix ensembles at the cost of more involved expressions. Finally, despite the seemingly strong assumption of rotation-invariant noise, our theory empirically predicts algorithmic performance on real data, pointing at strong universality properties.
Publishing Year
Date Published
2023-07-25
Journal Title
Proceedings of the National Academy of Sciences of the United States of America
Publisher
National Academy of Sciences
Acknowledgement
J.B. was funded by the European Union (ERC, CHORAL, project number 101039794). Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Research Council. Neither the European Union nor the granting authority can be held responsible for them. M.M. was supported by the 2019 Lopez-Loreta Prize. We would like to thank the reviewers for the insightful comments and, in particular, for suggesting the BAMP-inspired denoisers leading to AMP-AP.
Volume
120
Issue
30
Article Number
e2302028120
eISSN
IST-REx-ID
Cite this
Barbier J, Camilli F, Mondelli M, Sáenz M. Fundamental limits in structured principal component analysis and how to reach them. Proceedings of the National Academy of Sciences of the United States of America. 2023;120(30). doi:10.1073/pnas.2302028120
Barbier, J., Camilli, F., Mondelli, M., & Sáenz, M. (2023). Fundamental limits in structured principal component analysis and how to reach them. Proceedings of the National Academy of Sciences of the United States of America. National Academy of Sciences. https://doi.org/10.1073/pnas.2302028120
Barbier, Jean, Francesco Camilli, Marco Mondelli, and Manuel Sáenz. “Fundamental Limits in Structured Principal Component Analysis and How to Reach Them.” Proceedings of the National Academy of Sciences of the United States of America. National Academy of Sciences, 2023. https://doi.org/10.1073/pnas.2302028120.
J. Barbier, F. Camilli, M. Mondelli, and M. Sáenz, “Fundamental limits in structured principal component analysis and how to reach them,” Proceedings of the National Academy of Sciences of the United States of America, vol. 120, no. 30. National Academy of Sciences, 2023.
Barbier J, Camilli F, Mondelli M, Sáenz M. 2023. Fundamental limits in structured principal component analysis and how to reach them. Proceedings of the National Academy of Sciences of the United States of America. 120(30), e2302028120.
Barbier, Jean, et al. “Fundamental Limits in Structured Principal Component Analysis and How to Reach Them.” Proceedings of the National Academy of Sciences of the United States of America, vol. 120, no. 30, e2302028120, National Academy of Sciences, 2023, doi:10.1073/pnas.2302028120.
All files available under the following license(s):
Creative Commons Attribution 4.0 International Public License (CC-BY 4.0):
Main File(s)
File Name
2023_PNAS_Barbier.pdf
995.93 KB
Access Level
Open Access
Date Uploaded
2023-07-31
MD5 Checksum
1fc06228afdb3aa80cf8e7766bcf9dc5
Export
Marked PublicationsOpen Data ISTA Research Explorer
Sources
PMID: 37463204
PubMed | Europe PMC