---
OA_place: publisher
_id: '22857'
abstract:
- lang: eng
  text: "Artificial intelligence and machine learning have undergone an unprecedented
    evolution in the past decade, motivating a research effort toward a theory able
    to capture the qualitative behavior of large-scale neural systems. A central puzzle
    has been the clear benefit of scaling architecture size and overfitting the training
    set in supervised learning tasks. This evidence, in apparent contradiction with
    classical statistical learning theory, pushed researchers to develop a new theory
    capturing the interplay between the algorithmic and architectural bias of training
    and the specific target function, differently from previous methods rooted in
    uniform stability.\r\nThis approach has enabled a grounded understanding of novel
    learning regimes, typically through formal limits where the number of training
    samples $n$, data dimensions $d$, and model parameters $p$ grow to infinity at
    different rates. \\\\\r\nIn this thesis, we follow this approach, focusing on
    the trustworthiness of high-dimensional models: properties that are difficult
    to control during training or deployment and often emerge under unpredictable
    or adversarial conditions. In such settings, it is crucial to formally ensure
    a priori the reliability of machine learning systems.\r\nFirst, we study data
    memorization, both as label fitting and as the storage of private information
    about training samples in trained parameters. We prove that $p = \\Omega(n)$ parameters
    are sufficient for a deep neural network to memorize a generic set of labels,
    and for a model to memorize spurious features across training data. We then give
    evidence that $p = \\Omega(dn)$ parameters are instead necessary for an adversary
    to reconstruct the full training set from the trained parameters.\r\nSecond, we
    study robustness, both to adversarial perturbations and to distribution shift.
    We first prove that $p = \\Omega(dn)$ parameters can be sufficient for a class
    of neural networks to overfit the training data while guaranteeing robustness
    to adversarial perturbations. Then, we focus on spurious correlations learning
    in high-dimensional regression, studying the effect of the ridge regularization
    parameter in the proportional regime $n = \\Theta(d)$, and connecting it via an
    equivalence argument to the role of over-parameterization $p = \\Omega(n)$ in
    neural networks. We also investigate the architectural bias of attention-based
    networks, showing that they are sensitive to the replacement of individual words
    in an embedded sentence, allowing them to generalize on sentences where the contextual
    meaning depends on one or few words.\r\nFinally, we study differentially private
    optimization in high-dimensional regimes. We prove that standard private gradient
    methods do not suffer in the over-parameterized regime $p = \\Omega(n)$, challenging
    the current wisdom based on stability-derived generalization bounds. We then consider
    linear regression in the proportional regime $n = \\Theta(d)$, showing that standard
    private gradient descent can achieve optimal rates under appropriate hyper-parameter
    scaling, such as sufficiently small gradient clipping constants, whose role is
    still debated in practice."
acknowledged_ssus:
- _id: ScienComp
acknowledgement: "This project was partially supported by the 2019 Lopez-Loreta prize,\r\nthe
  European Union (ERC, INF2\r\n, project number 101161364), the Austrian Science Fund\r\n(FWF)
  10.55776/COE12, and a Google PhD fellowship in machine intelligence. Furthermore,\r\nthe
  candidate acknowledges the support from the Scientific Service Units of the Institute
  of\r\nScience and Technology Austria through resources provided by Scientific Computing."
alternative_title:
- ISTA Thesis
article_processing_charge: No
author:
- first_name: Simone
  full_name: Bombari, Simone
  id: ca726dda-de17-11ea-bc14-f9da834f63aa
  last_name: Bombari
citation:
  ama: Bombari S. Trustworthy machine learning in high dimensions. 2026. doi:<a href="https://doi.org/10.15479/AT-ISTA-22857">10.15479/AT-ISTA-22857</a>
  apa: Bombari, S. (2026). <i>Trustworthy machine learning in high dimensions</i>.
    Institute of Science and Technology Austria. <a href="https://doi.org/10.15479/AT-ISTA-22857">https://doi.org/10.15479/AT-ISTA-22857</a>
  chicago: Bombari, Simone. “Trustworthy Machine Learning in High Dimensions.” Institute
    of Science and Technology Austria, 2026. <a href="https://doi.org/10.15479/AT-ISTA-22857">https://doi.org/10.15479/AT-ISTA-22857</a>.
  ieee: S. Bombari, “Trustworthy machine learning in high dimensions,” Institute of
    Science and Technology Austria, 2026.
  ista: Bombari S. 2026. Trustworthy machine learning in high dimensions. Institute
    of Science and Technology Austria.
  mla: Bombari, Simone. <i>Trustworthy Machine Learning in High Dimensions</i>. Institute
    of Science and Technology Austria, 2026, doi:<a href="https://doi.org/10.15479/AT-ISTA-22857">10.15479/AT-ISTA-22857</a>.
  short: S. Bombari, Trustworthy Machine Learning in High Dimensions, Institute of
    Science and Technology Austria, 2026.
corr_author: '1'
das_tickbox: '0'
date_created: 2026-09-08T13:40:08Z
date_published: 2026-09-08T00:00:00Z
date_updated: 2026-09-21T13:07:01Z
day: '08'
ddc:
- '519'
degree_awarded: PhD
department:
- _id: GradSch
- _id: MaMo
doi: 10.15479/AT-ISTA-22857
doi_confirm: '1'
file:
- access_level: closed
  checksum: 8eab99d6dc6e4476826bdfc7c00fdebb
  content_type: application/zip
  creator: sbombari
  date_created: 2026-09-08T13:28:38Z
  date_updated: 2026-09-08T13:28:38Z
  file_id: '22859'
  file_name: Thesis copy.zip
  file_size: 99154325
  relation: source_file
- access_level: open_access
  checksum: 007033bafe4622ff4c2e758301354df1
  content_type: application/pdf
  creator: sbombari
  date_created: 2026-09-10T10:03:48Z
  date_updated: 2026-09-10T10:03:48Z
  file_id: '22898'
  file_name: 2026_Bombari_Simone_Thesis.pdf
  file_size: 12856211
  relation: main_file
  success: 1
file_date_updated: 2026-09-10T10:03:48Z
fulldoi: https://doi.org/10.15479/AT-ISTA-22857
has_accepted_license: '1'
keyword:
- machine learning
- high-dimensional statistics
- deep learning theory
- privacy
- memorization
- robustness
language:
- iso: eng
month: '09'
oa: 1
oa_version: Published Version
page: '446'
project:
- _id: 92099302-16d5-11f0-9cad-f9a785f54fbd
  name: 'Trustworthy Deep Learning Theory: Private Over-Parameterized Models and Robust
    LLMs'
- _id: 911e6d1f-16d5-11f0-9cad-c5c68c6a1cdf
  grant_number: '101161364'
  name: 'Inference in High Dimensions: Light-speed Algorithms and Information Limits'
- _id: 74caaef7-b034-11f1-8f2d-e0e993bb422e
  grant_number: COE12
  name: Bilateral Artificial Intelligence (Mondelli)
publication_identifier:
  isbn:
  - 978-3-99078-091-6
  issn:
  - 2663-337X
publication_status: published
publisher: Institute of Science and Technology Austria
related_material:
  record:
  - id: '12537'
    relation: part_of_dissertation
    status: public
  - id: '18972'
    relation: part_of_dissertation
    status: public
  - id: '18973'
    relation: part_of_dissertation
    status: public
  - id: '21324'
    relation: part_of_dissertation
    status: public
  - id: '22894'
    relation: part_of_dissertation
    status: public
  - id: '19627'
    relation: part_of_dissertation
    status: public
  - id: '12859'
    relation: part_of_dissertation
    status: public
researchdata_availability: no
status: public
supervisor:
- first_name: Marco
  full_name: Mondelli, Marco
  id: 27EB676C-8706-11E9-9510-7717E6697425
  last_name: Mondelli
  orcid: 0000-0002-3242-7020
supplementarymaterial: no
title: Trustworthy machine learning in high dimensions
type: dissertation
user_id: 8b945eb4-e2f2-11eb-945a-df72226e66a9
year: '2026'
...