---
OA_place: publisher
OA_type: gold
_id: '21328'
abstract:
- lang: eng
  text: Multi-index models provide a popular framework to investigate the learnability
    of functions with low-dimensional structure and, also due to their connections
    with neural networks, they have been object of recent intensive study. In this
    paper, we focus on recovering the subspace spanned by the signals via spectral
    estimators – a family of methods routinely used in practice, often as a warm-start
    for iterative algorithms. Our main technical contribution is a precise asymptotic
    characterization of the performance of spectral methods, when sample size and
    input dimension grow proportionally and the dimension p of the space to recover
    is fixed. Specifically, we locate the top-p eigenvalues of the spectral matrix
    and establish the overlaps between the corresponding eigenvectors (which give
    the spectral estimators) and a basis of the signal subspace. Our analysis unveils
    a phase transition phenomenon in which, as the sample complexity grows, eigenvalues
    escape from the bulk of the spectrum and, when that happens, eigenvectors recover
    directions of the desired subspace. The precise characterization we put forward
    enables the optimization of the data preprocessing, thus allowing to identify
    the spectral estimator that requires the minimal sample size for weak recovery.
acknowledgement: "This work was done when Y. Z. was at the Institute of Science and
  Technology Austria. Y. Z. and\r\nM. M. are funded by the European Union (ERC, INF2,
  project number 101161364). Views and\r\nopinions expressed are however those of
  the author(s) only and do not necessarily reflect those of the European Union or
  the European Research Council Executive Agency. Neither the European Union nor the
  granting authority can be held responsible for them. The authors would like to acknowledge
  (in alphabetical order) discussions with Yatin Dandi, Leonardo Defilippis and Bruno
  Loureiro concerning their parallel work (Defilippis et al., 2025)."
alternative_title:
- PMLR
article_processing_charge: No
arxiv: 1
author:
- first_name: Filip
  full_name: Kovačević, Filip
  id: d0258e7b-50b8-11ef-ad56-8b9f537b6b1b
  last_name: Kovačević
- first_name: Zhang
  full_name: Yihan, Zhang
  last_name: Yihan
- first_name: Marco
  full_name: Mondelli, Marco
  id: 27EB676C-8706-11E9-9510-7717E6697425
  last_name: Mondelli
  orcid: 0000-0002-3242-7020
citation:
  ama: 'Kovačević F, Yihan Z, Mondelli M. Spectral estimators for multi-index models:
    Precise asymptotics and optimal weak recovery. In: <i>Proceedings of 38th Conference
    on Learning Theory</i>. Vol 291. ML Research Press; 2025:3354-3404.'
  apa: 'Kovačević, F., Yihan, Z., &#38; Mondelli, M. (2025). Spectral estimators for
    multi-index models: Precise asymptotics and optimal weak recovery. In <i>Proceedings
    of 38th Conference on Learning Theory</i> (Vol. 291, pp. 3354–3404). Lyon, France:
    ML Research Press.'
  chicago: 'Kovačević, Filip, Zhang Yihan, and Marco Mondelli. “Spectral Estimators
    for Multi-Index Models: Precise Asymptotics and Optimal Weak Recovery.” In <i>Proceedings
    of 38th Conference on Learning Theory</i>, 291:3354–3404. ML Research Press, 2025.'
  ieee: 'F. Kovačević, Z. Yihan, and M. Mondelli, “Spectral estimators for multi-index
    models: Precise asymptotics and optimal weak recovery,” in <i>Proceedings of 38th
    Conference on Learning Theory</i>, Lyon, France, 2025, vol. 291, pp. 3354–3404.'
  ista: 'Kovačević F, Yihan Z, Mondelli M. 2025. Spectral estimators for multi-index
    models: Precise asymptotics and optimal weak recovery. Proceedings of 38th Conference
    on Learning Theory. COLT: Conference on Learning Theory, PMLR, vol. 291, 3354–3404.'
  mla: 'Kovačević, Filip, et al. “Spectral Estimators for Multi-Index Models: Precise
    Asymptotics and Optimal Weak Recovery.” <i>Proceedings of 38th Conference on Learning
    Theory</i>, vol. 291, ML Research Press, 2025, pp. 3354–404.'
  short: F. Kovačević, Z. Yihan, M. Mondelli, in:, Proceedings of 38th Conference
    on Learning Theory, ML Research Press, 2025, pp. 3354–3404.
conference:
  end_date: 2025-07-04
  location: Lyon, France
  name: 'COLT: Conference on Learning Theory'
  start_date: 2025-06-30
corr_author: '1'
date_created: 2026-02-18T12:12:47Z
date_published: 2025-07-01T00:00:00Z
date_updated: 2026-02-19T09:03:53Z
day: '01'
ddc:
- '000'
department:
- _id: MaMo
external_id:
  arxiv:
  - '2502.01583'
file:
- access_level: open_access
  checksum: 19aa70ab4f57fb9067b6ebb99a5fd6f0
  content_type: application/pdf
  creator: dernst
  date_created: 2026-02-19T09:03:43Z
  date_updated: 2026-02-19T09:03:43Z
  file_id: '21339'
  file_name: 2025_LearningTheory_Kovacevic.pdf
  file_size: 844611
  relation: main_file
  success: 1
file_date_updated: 2026-02-19T09:03:43Z
has_accepted_license: '1'
intvolume: '       291'
language:
- iso: eng
license: https://creativecommons.org/licenses/by/4.0/
month: '07'
oa: 1
oa_version: Published Version
page: 3354-3404
project:
- _id: 911e6d1f-16d5-11f0-9cad-c5c68c6a1cdf
  grant_number: '101161364'
  name: 'Inference in High Dimensions: Light-speed Algorithms and Information Limits'
publication: Proceedings of 38th Conference on Learning Theory
publication_identifier:
  eissn:
  - 2640-3498
publication_status: published
publisher: ML Research Press
quality_controlled: '1'
scopus_import: '1'
status: public
title: 'Spectral estimators for multi-index models: Precise asymptotics and optimal
  weak recovery'
tmp:
  image: /images/cc_by.png
  legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode
  name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)
  short: CC BY (4.0)
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 291
year: '2025'
...
---
OA_place: repository
OA_type: green
_id: '20300'
abstract:
- lang: eng
  text: Simultaneously addressing multiple objectives is becoming increasingly important
    in modern machine learning. At the same time, data is often high-dimensional and
    costly to label. For a single objective such as prediction risk, conventional
    regularization techniques are known to improve generalization when the data exhibits
    low-dimensional structure like sparsity. However, it is largely unexplored how
    to leverage this structure in the context of multi-objective learning (MOL) with
    multiple competing objectives. In this work, we discuss how the application of
    vanilla regularization approaches can fail, and propose a two-stage MOL framework
    that can successfully leverage low-dimensional structure. We demonstrate its effectiveness
    experimentally for multi-distribution learning and fairness-risk trade-offs.
acknowledgement: "We thank Junhyung Park for valuable feedback on the manuscript.
  AT was supported by a PhD fellowship from the Swiss Data Science Center. TW was
  supported by the SNF Grant 204439. This work was done in part while TW and FY were
  visiting the Simons Institute for the Theory of\r\nComputing."
alternative_title:
- PMLR
article_processing_charge: No
arxiv: 1
author:
- first_name: Tobias
  full_name: Wegel, Tobias
  last_name: Wegel
- first_name: Filip
  full_name: Kovačević, Filip
  id: d0258e7b-50b8-11ef-ad56-8b9f537b6b1b
  last_name: Kovačević
- first_name: Alexandru
  full_name: Ţifrea, Alexandru
  last_name: Ţifrea
- first_name: Fanny
  full_name: Yang, Fanny
  last_name: Yang
citation:
  ama: 'Wegel T, Kovačević F, Ţifrea A, Yang F. Learning Pareto manifolds in high
    dimensions: How can regularization help? In: <i>The 28th International Conference
    on Artificial Intelligence and Statistics</i>. Vol 258. ML Research Press; 2025:4591-4599.'
  apa: 'Wegel, T., Kovačević, F., Ţifrea, A., &#38; Yang, F. (2025). Learning Pareto
    manifolds in high dimensions: How can regularization help? In <i>The 28th International
    Conference on Artificial Intelligence and Statistics</i> (Vol. 258, pp. 4591–4599).
    Mai Khao, Thailand: ML Research Press.'
  chicago: 'Wegel, Tobias, Filip Kovačević, Alexandru Ţifrea, and Fanny Yang. “Learning
    Pareto Manifolds in High Dimensions: How Can Regularization Help?” In <i>The 28th
    International Conference on Artificial Intelligence and Statistics</i>, 258:4591–99.
    ML Research Press, 2025.'
  ieee: 'T. Wegel, F. Kovačević, A. Ţifrea, and F. Yang, “Learning Pareto manifolds
    in high dimensions: How can regularization help?,” in <i>The 28th International
    Conference on Artificial Intelligence and Statistics</i>, Mai Khao, Thailand,
    2025, vol. 258, pp. 4591–4599.'
  ista: 'Wegel T, Kovačević F, Ţifrea A, Yang F. 2025. Learning Pareto manifolds in
    high dimensions: How can regularization help? The 28th International Conference
    on Artificial Intelligence and Statistics. AISTATS: Conference on Artificial Intelligence
    and Statistics, PMLR, vol. 258, 4591–4599.'
  mla: 'Wegel, Tobias, et al. “Learning Pareto Manifolds in High Dimensions: How Can
    Regularization Help?” <i>The 28th International Conference on Artificial Intelligence
    and Statistics</i>, vol. 258, ML Research Press, 2025, pp. 4591–99.'
  short: T. Wegel, F. Kovačević, A. Ţifrea, F. Yang, in:, The 28th International Conference
    on Artificial Intelligence and Statistics, ML Research Press, 2025, pp. 4591–4599.
conference:
  end_date: 2025-05-05
  location: Mai Khao, Thailand
  name: 'AISTATS: Conference on Artificial Intelligence and Statistics'
  start_date: 2025-05-03
date_created: 2025-09-07T22:01:35Z
date_published: 2025-05-01T00:00:00Z
date_updated: 2025-09-09T07:00:34Z
day: '01'
department:
- _id: MaMo
external_id:
  arxiv:
  - '2503.08849'
intvolume: '       258'
language:
- iso: eng
main_file_link:
- open_access: '1'
  url: https://doi.org/10.48550/arXiv.2503.08849
month: '05'
oa: 1
oa_version: Preprint
page: 4591-4599
publication: The 28th International Conference on Artificial Intelligence and Statistics
publication_identifier:
  eissn:
  - 2640-3498
publication_status: published
publisher: ML Research Press
quality_controlled: '1'
scopus_import: '1'
status: public
title: 'Learning Pareto manifolds in high dimensions: How can regularization help?'
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 258
year: '2025'
...