Estimation in rotationally invariant generalized linear models via approximate message passing

conference paper Estimation in rotationally invariant generalized linear models via approximate message passing published yes Ramji Venkataramanan author Kevin Kögler author 94ec913c-dc85-11ea-9058-e5051ab2428b Marco Mondelli author 27EB676C-8706-11E9-9510-7717E66974250000-0002-3242-7020 MaMo department ICML: International Conference on Machine Learning Prix Lopez-Loretta 2019 - Marco Mondelli project We consider the problem of signal estimation in generalized linear models defined via rotationally invariant design matrices. Since these matrices can have an arbitrary spectral distribution, this model is well suited for capturing complex correlation structures which often arise in applications. We propose a novel family of approximate message passing (AMP) algorithms for signal estimation, and rigorously characterize their performance in the high-dimensional limit via a state evolution recursion. Our rotationally invariant AMP has complexity of the same order as the existing AMP derived under the restrictive assumption of a Gaussian design; our algorithm also recovers this existing AMP as a special case. Numerical results showcase a performance close to Vector AMP (which is conjectured to be Bayes-optimal in some settings), but obtained with a much lower complexity, as the proposed algorithm does not require a computationally expensive singular value decomposition. https://research-explorer.ista.ac.at/download/12540/12547/2022_PMLR_Venkataramanan.pdf application/pdfno ML Research Press2022Baltimore, MD, United States eng Proceedings of the 39th International Conference on Machine Learning 162 Venkataramanan, Ramji, et al. “Estimation in Rotationally Invariant Generalized Linear Models via Approximate Message Passing.” Proceedings of the 39th International Conference on Machine Learning, vol. 162, 22, ML Research Press, 2022. R. Venkataramanan, K. Kögler, M. Mondelli, in:, Proceedings of the 39th International Conference on Machine Learning, ML Research Press, 2022. Venkataramanan, Ramji, Kevin Kögler, and Marco Mondelli. “Estimation in Rotationally Invariant Generalized Linear Models via Approximate Message Passing.” In Proceedings of the 39th International Conference on Machine Learning, Vol. 162. ML Research Press, 2022. R. Venkataramanan, K. Kögler, and M. Mondelli, “Estimation in rotationally invariant generalized linear models via approximate message passing,” in Proceedings of the 39th International Conference on Machine Learning, Baltimore, MD, United States, 2022, vol. 162. Venkataramanan R, Kögler K, Mondelli M. Estimation in rotationally invariant generalized linear models via approximate message passing. In: Proceedings of the 39th International Conference on Machine Learning. Vol 162. ML Research Press; 2022. Venkataramanan, R., Kögler, K., & Mondelli, M. (2022). Estimation in rotationally invariant generalized linear models via approximate message passing. In Proceedings of the 39th International Conference on Machine Learning (Vol. 162). Baltimore, MD, United States: ML Research Press. Venkataramanan R, Kögler K, Mondelli M. 2022. Estimation in rotationally invariant generalized linear models via approximate message passing. Proceedings of the 39th International Conference on Machine Learning. ICML: International Conference on Machine Learning vol. 162, 22. 125402023-02-10T13:49:04Z2025-04-15T07:50:16Z