---
OA_place: publisher
OA_type: gold
_id: '22894'
abstract:
- lang: eng
text: "Large-scale deep learning models are known to memorize parts of the training\r\nset.
In machine learning theory, memorization is often framed as interpolation or\r\nlabel
fitting, and classical results show that this can be achieved when the number\r\nof
parameters p in the model is larger than the number of training samples n. In\r\nthis
work, we consider memorization from the perspective of data reconstruction,\r\ndemonstrating
that this can be achieved when p is larger than dn, where d is\r\nthe dimensionality
of the data. More specifically, we show that, in the random\r\nfeatures model,
when p ≫ dn, the subspace spanned by the training samples in\r\nfeature space
gives sufficient information to identify the individual samples in input\r\nspace.
Our analysis suggests an optimization method to reconstruct the dataset\r\nfrom
the model parameters, and we demonstrate that this method performs well on\r\nvarious
architectures (random features, two-layer fully-connected and deep residual\r\nnetworks).
Our results reveal a law of data reconstruction, according to which the\r\nentire
training dataset can be recovered as p exceeds the threshold dn.\r\n"
acknowledgement: "M.M. is funded by the European Union (ERC, INF2\r\n, project number
101161364). S.B. was supported\r\nby a Google PhD fellowship. L.I. acknowledges
the grant received from the European Union NextGenerationEU (Piano Nazionale di
Ripresa E Resilienza (PNRR)) DM 351 on Trustworthy AI. T.T. &\r\nL.I. acknowledge
the EU project ELSA - European Lighthouse on Secure and Safe AI. This study was\r\ncarried
out within the FAIR - Future Artificial Intelligence Research and received funding
from the\r\nEuropean Union Next-GenerationEU (PIANO NAZIONALE DI RIPRESA E RESILIENZA
(PNRR)\r\n– MISSIONE 4 COMPONENTE 2, INVESTIMENTO 1.3 – D.D. 1555 11/10/2022, PE00000013).\r\nThis
manuscript reflects only the authors’ views and opinions, neither the European Union
nor the\r\nEuropean Commission can be considered responsible for them. The authors
would like to thank\r\nYizhe Zhu for helpful discussions."
article_processing_charge: No
arxiv: 1
author:
- first_name: Leonardo
full_name: Iurada, Leonardo
last_name: Iurada
- first_name: Simone
full_name: Bombari, Simone
id: ca726dda-de17-11ea-bc14-f9da834f63aa
last_name: Bombari
- first_name: Tatiana
full_name: Tommasi, Tatiana
last_name: Tommasi
- first_name: Marco
full_name: Mondelli, Marco
id: 27EB676C-8706-11E9-9510-7717E6697425
last_name: Mondelli
orcid: 0000-0002-3242-7020
citation:
ama: 'Iurada L, Bombari S, Tommasi T, Mondelli M. A law of data reconstruction for
random features (and beyond). In: 14th International Conference on Learning
Representations. Vol 2026. OpenReview; 2026:145275-145314.'
apa: 'Iurada, L., Bombari, S., Tommasi, T., & Mondelli, M. (2026). A law of
data reconstruction for random features (and beyond). In 14th International
Conference on Learning Representations (Vol. 2026, pp. 145275–145314). Rio
de Janeiro, Brazil: OpenReview.'
chicago: Iurada, Leonardo, Simone Bombari, Tatiana Tommasi, and Marco Mondelli.
“A Law of Data Reconstruction for Random Features (and Beyond).” In 14th International
Conference on Learning Representations, 2026:145275–314. OpenReview, 2026.
ieee: L. Iurada, S. Bombari, T. Tommasi, and M. Mondelli, “A law of data reconstruction
for random features (and beyond),” in 14th International Conference on Learning
Representations, Rio de Janeiro, Brazil, 2026, vol. 2026, pp. 145275–145314.
ista: 'Iurada L, Bombari S, Tommasi T, Mondelli M. 2026. A law of data reconstruction
for random features (and beyond). 14th International Conference on Learning Representations.
ICLR: International Conference on Learning Representations vol. 2026, 145275–145314.'
mla: Iurada, Leonardo, et al. “A Law of Data Reconstruction for Random Features
(and Beyond).” 14th International Conference on Learning Representations,
vol. 2026, OpenReview, 2026, pp. 145275–314.
short: L. Iurada, S. Bombari, T. Tommasi, M. Mondelli, in:, 14th International Conference
on Learning Representations, OpenReview, 2026, pp. 145275–145314.
conference:
end_date: 2026-04-27
location: Rio de Janeiro, Brazil
name: 'ICLR: International Conference on Learning Representations '
start_date: 2026-04-23
corr_author: '1'
das_tickbox: '0'
date_created: 2026-09-09T13:31:46Z
date_published: 2026-01-26T00:00:00Z
date_updated: 2026-09-11T12:26:43Z
day: '26'
ddc:
- '000'
department:
- _id: GradSch
- _id: MaMo
external_id:
arxiv:
- '2509.22214'
file:
- access_level: open_access
checksum: a3dcba6649b7496124fa0ae68bf2c4cb
content_type: application/pdf
creator: cchlebak
date_created: 2026-09-11T12:07:48Z
date_updated: 2026-09-11T12:07:48Z
file_id: '22907'
file_name: 2026_ICLR_Iurada.pdf
file_size: 7869510
relation: main_file
success: 1
file_date_updated: 2026-09-11T12:07:48Z
has_accepted_license: '1'
intvolume: ' 2026'
language:
- iso: eng
month: '01'
oa: 1
oa_version: Published Version
page: 145275-145314
project:
- _id: 911e6d1f-16d5-11f0-9cad-c5c68c6a1cdf
grant_number: '101161364'
name: 'Inference in High Dimensions: Light-speed Algorithms and Information Limits'
- _id: 92099302-16d5-11f0-9cad-f9a785f54fbd
name: 'Trustworthy Deep Learning Theory: Private Over-Parameterized Models and Robust
LLMs'
publication: 14th International Conference on Learning Representations
publication_identifier:
isbn:
- '9798331339678'
publication_status: published
publisher: OpenReview
quality_controlled: '1'
related_material:
record:
- id: '22857'
relation: dissertation_contains
status: for_moderation
status: public
title: A law of data reconstruction for random features (and beyond)
tmp:
image: /images/cc_by.png
legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode
name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)
short: CC BY (4.0)
type: conference
user_id: 8b945eb4-e2f2-11eb-945a-df72226e66a9
volume: 2026
year: '2026'
...