---
OA_place: publisher
OA_type: gold
_id: '22826'
abstract:
- lang: eng
text: "Large Language Models (LLMs) have demonstrated the ability to tackle increasingly
complex tasks through advanced reasoning, long-form content generation,\r\nand
tool use. Solving these tasks often involves long inference-time computations.\r\nIn
human problem solving, a common strategy to expedite work is collaboration: by\r\ndividing
the problem into sub-tasks, exploring different strategies concurrently, etc.\r\nRecent
research has shown that LLMs can also operate in parallel by implementing\r\nexplicit
cooperation frameworks, such as voting mechanisms or the explicit creation of
independent sub-tasks that can be executed in parallel. However, each of\r\nthese
frameworks may not be suitable for all types of tasks, which can hinder their\r\napplicability.
In this work, we propose a different design approach: we run LLM\r\n“workers”
in parallel , allowing them to synchronize via a concurrently-updated\r\nattention
cache and prompt these workers to decide how best to collaborate. Our\r\napproach
allows the LLM instances to come up with their own collaboration strategy for
the problem at hand, all the while “seeing” each other’s memory in the\r\nconcurrent
KV cache. We implement this approach via Hogwild! Inference: a\r\nparallel LLM
inference engine where multiple instances of the same LLM run in\r\nparallel with
the same attention cache, with “instant” access to each other’s memory.1 Hogwild!
Inference takes advantage of Rotary Position Embeddings (RoPE)\r\nto avoid recomputation
while improving parallel hardware utilization. We find that\r\nmodern reasoning-capable
LLMs can perform inference with shared Key-Value\r\ncache out of the box, without
additional fine-tuning."
acknowledgement: "We thank Vladimir Malinovskii for his help with brainstorming, helpful\r\nfeedback
and suggesting future work directions. We also thank Philip Zmushko for proofreading."
alternative_title:
- Advances in Neural Information Processing Systems
article_processing_charge: No
author:
- first_name: Gleb
full_name: Rodionov, Gleb
last_name: Rodionov
- first_name: Roman
full_name: Garipov, Roman
last_name: Garipov
- first_name: Alina
full_name: Shutova, Alina
last_name: Shutova
- first_name: George
full_name: Yakushev, George
last_name: Yakushev
- first_name: Erik
full_name: Schultheis, Erik
id: 2786b299-e6b0-11f0-91da-9243fe3ef96b
last_name: Schultheis
- first_name: Vage
full_name: Egiazarian, Vage
id: 77451e76-92b2-11ef-a4d1-8dbaa06e16ad
last_name: Egiazarian
- first_name: Anton
full_name: Sinitsin, Anton
last_name: Sinitsin
- first_name: Denis
full_name: Kuznedelev, Denis
last_name: Kuznedelev
- first_name: Dan-Adrian
full_name: Alistarh, Dan-Adrian
id: 4A899BFC-F248-11E8-B48F-1D18A9856A87
last_name: Alistarh
orcid: 0000-0003-3650-940X
citation:
ama: 'Rodionov G, Garipov R, Shutova A, et al. Hogwild! Inference: Parallel LLM
generation via concurrent attention. In: 39th Conference on Neural Information
Processing Systems. Vol 38. Neural Information Processing Systems Foundation;
2025:52014-52055. doi:10.52202/085713-1551'
apa: 'Rodionov, G., Garipov, R., Shutova, A., Yakushev, G., Schultheis, E., Egiazarian,
V., … Alistarh, D.-A. (2025). Hogwild! Inference: Parallel LLM generation via
concurrent attention. In 39th Conference on Neural Information Processing Systems
(Vol. 38, pp. 52014–52055). San Diego, CA, United States: Neural Information Processing
Systems Foundation. https://doi.org/10.52202/085713-1551'
chicago: 'Rodionov, Gleb, Roman Garipov, Alina Shutova, George Yakushev, Erik Schultheis,
Vage Egiazarian, Anton Sinitsin, Denis Kuznedelev, and Dan-Adrian Alistarh. “Hogwild!
Inference: Parallel LLM Generation via Concurrent Attention.” In 39th Conference
on Neural Information Processing Systems, 38:52014–55. Neural Information
Processing Systems Foundation, 2025. https://doi.org/10.52202/085713-1551.'
ieee: 'G. Rodionov et al., “Hogwild! Inference: Parallel LLM generation via
concurrent attention,” in 39th Conference on Neural Information Processing
Systems, San Diego, CA, United States, 2025, vol. 38, pp. 52014–52055.'
ista: 'Rodionov G, Garipov R, Shutova A, Yakushev G, Schultheis E, Egiazarian V,
Sinitsin A, Kuznedelev D, Alistarh D-A. 2025. Hogwild! Inference: Parallel LLM
generation via concurrent attention. 39th Conference on Neural Information Processing
Systems. NeurIPS: Neural Information Processing Systems, Advances in Neural Information
Processing Systems, vol. 38, 52014–52055.'
mla: 'Rodionov, Gleb, et al. “Hogwild! Inference: Parallel LLM Generation via Concurrent
Attention.” 39th Conference on Neural Information Processing Systems, vol.
38, Neural Information Processing Systems Foundation, 2025, pp. 52014–55, doi:10.52202/085713-1551.'
short: G. Rodionov, R. Garipov, A. Shutova, G. Yakushev, E. Schultheis, V. Egiazarian,
A. Sinitsin, D. Kuznedelev, D.-A. Alistarh, in:, 39th Conference on Neural Information
Processing Systems, Neural Information Processing Systems Foundation, 2025, pp.
52014–52055.
conference:
end_date: 2025-12-07
location: San Diego, CA, United States
name: 'NeurIPS: Neural Information Processing Systems'
start_date: 2025-12-02
das_tickbox: '0'
date_created: 2026-09-06T22:01:59Z
date_published: 2025-12-02T00:00:00Z
date_updated: 2026-09-10T07:06:31Z
day: '02'
ddc:
- '000'
department:
- _id: DaAl
doi: 10.52202/085713-1551
fulldoi: https://doi.org/10.52202/085713-1551
has_accepted_license: '1'
intvolume: ' 38'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.52202/085713-1551
month: '12'
oa: 1
oa_version: Published Version
page: 52014-52055
publication: 39th Conference on Neural Information Processing Systems
publication_identifier:
isbn:
- '9798331338275'
issn:
- 1049-5258
publication_status: published
publisher: Neural Information Processing Systems Foundation
quality_controlled: '1'
researchdata_availability: no
scopus_import: '1'
status: public
supplementarymaterial: yes
title: 'Hogwild! Inference: Parallel LLM generation via concurrent attention'
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 38
year: '2025'
...
---
OA_place: publisher
OA_type: free access
_id: '22825'
abstract:
- lang: eng
text: "The empirical emergence of neural collapse—a surprising symmetry in the feature
representations of the training data in the penultimate layer of deep neural\r\nnetworks—has
spurred a line of theoretical research aimed at its understanding.\r\nHowever,
existing work focuses on data-agnostic models or, when data structure is\r\ntaken
into account, it remains limited to multi-layer perceptrons. Our paper fills\r\nboth
these gaps by analyzing modern architectures in a data-aware regime: we\r\nprove
that global optima of deep regularized transformers and residual networks\r\n(ResNets)
with LayerNorm trained with cross entropy or mean squared error loss\r\nare approximately
collapsed, and the approximation gets tighter as the depth grows.\r\nMore generally,
we formally reduce any end-to-end large-depth ResNet or transformer training into
an equivalent unconstrained features model, thus justifying its\r\nwide use in
the literature even beyond data-agnostic settings. Our theoretical results\r\nare
supported by experiments on computer vision and language datasets showing\r\nthat,
as the depth grows, neural collapse indeed becomes more prominent."
acknowledged_ssus:
- _id: ScienComp
acknowledgement: "M. M. and P. S. are funded by the European Union (ERC, INF2\r\n,
project number 101161364). Views\r\nand opinions expressed are however those of
the author(s) only and do not necessarily reflect those\r\nof the European Union
or the European Research Council Executive Agency. Neither the European\r\nUnion
nor the granting authority can be held responsible for them. This research was supported\r\nby
the Scientific Service Units (SSU) of ISTA through resources provided by Scientific
Computing (SciComp)."
alternative_title:
- Advances in Neural Information Processing Systems
article_processing_charge: No
arxiv: 1
author:
- first_name: Peter
full_name: Súkeník, Peter
id: d64d6a8d-eb8e-11eb-b029-96fd216dec3c
last_name: Súkeník
- first_name: Christoph
full_name: Lampert, Christoph
id: 40C20FD2-F248-11E8-B48F-1D18A9856A87
last_name: Lampert
orcid: 0000-0001-8622-7887
- first_name: Marco
full_name: Mondelli, Marco
id: 27EB676C-8706-11E9-9510-7717E6697425
last_name: Mondelli
orcid: 0000-0002-3242-7020
citation:
ama: 'Súkeník P, Lampert C, Mondelli M. Neural collapse is globally optimal in deep
regularized ResNets and transformers. In: 39th Conference on Neural Information
Processing Systems. Vol 38. Neural Information Processing Systems Foundation;
2025:48646-48677. doi:10.52202/085713-1450'
apa: 'Súkeník, P., Lampert, C., & Mondelli, M. (2025). Neural collapse is globally
optimal in deep regularized ResNets and transformers. In 39th Conference on
Neural Information Processing Systems (Vol. 38, pp. 48646–48677). San Diego,
CA, United States: Neural Information Processing Systems Foundation. https://doi.org/10.52202/085713-1450'
chicago: Súkeník, Peter, Christoph Lampert, and Marco Mondelli. “Neural Collapse
Is Globally Optimal in Deep Regularized ResNets and Transformers.” In 39th
Conference on Neural Information Processing Systems, 38:48646–77. Neural Information
Processing Systems Foundation, 2025. https://doi.org/10.52202/085713-1450.
ieee: P. Súkeník, C. Lampert, and M. Mondelli, “Neural collapse is globally optimal
in deep regularized ResNets and transformers,” in 39th Conference on Neural
Information Processing Systems, San Diego, CA, United States, 2025, vol. 38,
pp. 48646–48677.
ista: 'Súkeník P, Lampert C, Mondelli M. 2025. Neural collapse is globally optimal
in deep regularized ResNets and transformers. 39th Conference on Neural Information
Processing Systems. NeurIPS: Neural Information Processing Systems, Advances in
Neural Information Processing Systems, vol. 38, 48646–48677.'
mla: Súkeník, Peter, et al. “Neural Collapse Is Globally Optimal in Deep Regularized
ResNets and Transformers.” 39th Conference on Neural Information Processing
Systems, vol. 38, Neural Information Processing Systems Foundation, 2025,
pp. 48646–77, doi:10.52202/085713-1450.
short: P. Súkeník, C. Lampert, M. Mondelli, in:, 39th Conference on Neural Information
Processing Systems, Neural Information Processing Systems Foundation, 2025, pp.
48646–48677.
conference:
end_date: 2025-12-07
location: San Diego, CA, United States
name: 'NeurIPS: Neural Information Processing Systems'
start_date: 2025-12-02
corr_author: '1'
das_tickbox: '0'
date_created: 2026-09-06T22:01:59Z
date_published: 2025-12-02T00:00:00Z
date_updated: 2026-09-10T08:11:46Z
day: '02'
ddc:
- '000'
department:
- _id: MaMo
- _id: GradSch
- _id: ChLa
doi: 10.52202/085713-1450
external_id:
arxiv:
- '2505.15239'
fulldoi: https://doi.org/10.52202/085713-1450
intvolume: ' 38'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.52202/085713-1450
month: '12'
oa: 1
oa_version: Published Version
page: 48646-48677
project:
- _id: 911e6d1f-16d5-11f0-9cad-c5c68c6a1cdf
grant_number: '101161364'
name: 'Inference in High Dimensions: Light-speed Algorithms and Information Limits'
publication: 39th Conference on Neural Information Processing Systems
publication_identifier:
isbn:
- '9798331338275'
issn:
- 1049-5258
publication_status: published
publisher: Neural Information Processing Systems Foundation
quality_controlled: '1'
researchdata_availability: no
scopus_import: '1'
status: public
supplementarymaterial: no
title: Neural collapse is globally optimal in deep regularized ResNets and transformers
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 38
year: '2025'
...
---
OA_place: publisher
OA_type: free access
_id: '22824'
abstract:
- lang: eng
text: "We present new fast-rate PAC-Bayesian generalization bounds for multi-task
and\r\nmeta-learning in the unbalanced setting, i.e. when the tasks have training
sets of\r\ndifferent sizes, as is typically the case in real-world scenarios.
Previously, only\r\nstandard-rate bounds were known for this situation, while
fast-rate bounds were\r\nlimited to the setting where all training sets are of
equal size. Our new bounds\r\nare numerically computable as well as interpretable,
and we demonstrate their\r\nflexibility in handling a number of cases where they
give stronger guarantees\r\nthan previous bounds. Besides the bounds themselves,
we also make conceptual\r\ncontributions: we demonstrate that the unbalanced multi-task
setting has different\r\nstatistical properties than the balanced situation, specifically
that proofs from\r\nthe balanced situation do not carry over to the unbalanced
setting. Additionally,\r\nwe shed light on the fact that the unbalanced situation
allows two meaningful\r\ndefinitions of multi-task risk, depending on whether
all tasks should be considered\r\nequally important or if sample-rich tasks should
receive more weight than samplepoor ones."
acknowledged_ssus:
- _id: ScienComp
acknowledgement: This research was supported by the Scientific Service Units (SSU)
of ISTA through resources provided by Scientific Computing (SciComp).
alternative_title:
- Advances in Neural Information Processing Systems
article_processing_charge: No
author:
- first_name: Hossein
full_name: Zakerinia, Hossein
id: 653bd8b6-f394-11eb-9cf6-c0bbf6cd78d4
last_name: Zakerinia
orcid: 0009-0007-3977-6462
- first_name: Christoph
full_name: Lampert, Christoph
id: 40C20FD2-F248-11E8-B48F-1D18A9856A87
last_name: Lampert
orcid: 0000-0001-8622-7887
citation:
ama: 'Zakerinia H, Lampert C. Fast rate bounds for multi-task and meta-learning
with different sample sizes. In: 39th Conference on Neural Information Processing
Systems. Vol 38. Neural Information Processing Systems Foundation; 2025:9062-9093.
doi:10.52202/085713-0278'
apa: 'Zakerinia, H., & Lampert, C. (2025). Fast rate bounds for multi-task and
meta-learning with different sample sizes. In 39th Conference on Neural Information
Processing Systems (Vol. 38, pp. 9062–9093). San Diego, CA, United States:
Neural Information Processing Systems Foundation. https://doi.org/10.52202/085713-0278'
chicago: Zakerinia, Hossein, and Christoph Lampert. “Fast Rate Bounds for Multi-Task
and Meta-Learning with Different Sample Sizes.” In 39th Conference on Neural
Information Processing Systems, 38:9062–93. Neural Information Processing
Systems Foundation, 2025. https://doi.org/10.52202/085713-0278.
ieee: H. Zakerinia and C. Lampert, “Fast rate bounds for multi-task and meta-learning
with different sample sizes,” in 39th Conference on Neural Information Processing
Systems, San Diego, CA, United States, 2025, vol. 38, pp. 9062–9093.
ista: 'Zakerinia H, Lampert C. 2025. Fast rate bounds for multi-task and meta-learning
with different sample sizes. 39th Conference on Neural Information Processing
Systems. NeurIPS: Neural Information Processing Systems, Advances in Neural Information
Processing Systems, vol. 38, 9062–9093.'
mla: Zakerinia, Hossein, and Christoph Lampert. “Fast Rate Bounds for Multi-Task
and Meta-Learning with Different Sample Sizes.” 39th Conference on Neural Information
Processing Systems, vol. 38, Neural Information Processing Systems Foundation,
2025, pp. 9062–93, doi:10.52202/085713-0278.
short: H. Zakerinia, C. Lampert, in:, 39th Conference on Neural Information Processing
Systems, Neural Information Processing Systems Foundation, 2025, pp. 9062–9093.
conference:
end_date: 2025-12-07
location: San Diego, CA, United States
name: 'NeurIPS: Neural Information Processing Systems'
start_date: 2025-12-02
corr_author: '1'
das_tickbox: '0'
date_created: 2026-09-06T22:01:59Z
date_published: 2025-12-02T00:00:00Z
date_updated: 2026-09-10T08:37:33Z
day: '02'
department:
- _id: GradSch
- _id: ChLa
doi: 10.52202/085713-0278
fulldoi: https://doi.org/10.52202/085713-0278
intvolume: ' 38'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.52202/085713-0278
month: '12'
oa: 1
oa_version: Published Version
page: 9062-9093
publication: 39th Conference on Neural Information Processing Systems
publication_identifier:
isbn:
- '9798331338275'
issn:
- 1049-5258
publication_status: published
publisher: Neural Information Processing Systems Foundation
quality_controlled: '1'
researchdata_availability: no
scopus_import: '1'
status: public
supplementarymaterial: yes
title: Fast rate bounds for multi-task and meta-learning with different sample sizes
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 38
year: '2025'
...
---
OA_place: publisher
OA_type: gold
_id: '22823'
abstract:
- lang: eng
text: This position paper argues that setting the privacy budget in differential
privacy should not be viewed as an important limitation of differential privacy
compared to alternative methods for privacy-preserving machine learning. The so-called
problem of interpreting the privacy budget is often presented as a major hindrance
to the wider adoption of differential privacy in real-world deployments and is
sometimes used to promote alternative mitigation techniques for data protection.
We believe this misleads decision-makers into choosing unsafe methods. We argue
that the difficulty in interpreting privacy budgets does not stem from the definition
of differential privacy itself, but from the intrinsic difficulty of estimating
privacy risks in context, a challenge that any rigorous method for privacy risk
assessment face. Moreover, we claim that any sound method for estimating privacy
risks should, given the current state of research, be expressible within the differential
privacy framework or justify why it cannot.
acknowledgement: "This research was funded in whole by the Austrian Science Fund (FWF)
10.55776/COE12. I thank\r\nAlberto Naibo for the discussion and his invitation to
\"Logique, droit, IA : penser les algorithmes\"\r\ncolloquium. I thank Simone Bombari
for the discussions. I thank EurIPS for the active poster session."
alternative_title:
- Advances in Neural Information Processing Systems
article_processing_charge: No
author:
- first_name: Edwige Audrey Lucienne
full_name: Cyffers, Edwige Audrey Lucienne
id: 20d4c299-977a-11ef-ae55-98b15ac64a57
last_name: Cyffers
citation:
ama: 'Cyffers EAL. Setting ε is not the issue in differential privacy. In: 39th
Conference on Neural Information Processing Systems. Vol 38. Neural Information
Processing Systems Foundation; 2025:159016-159028. doi:10.52202/085713-4798'
apa: 'Cyffers, E. A. L. (2025). Setting ε is not the issue in differential privacy.
In 39th Conference on Neural Information Processing Systems (Vol. 38, pp.
159016–159028). San Diego, CA, United States: Neural Information Processing Systems
Foundation. https://doi.org/10.52202/085713-4798'
chicago: Cyffers, Edwige Audrey Lucienne. “Setting ε Is Not the Issue in Differential
Privacy.” In 39th Conference on Neural Information Processing Systems,
38:159016–28. Neural Information Processing Systems Foundation, 2025. https://doi.org/10.52202/085713-4798.
ieee: E. A. L. Cyffers, “Setting ε is not the issue in differential privacy,” in
39th Conference on Neural Information Processing Systems, San Diego, CA,
United States, 2025, vol. 38, pp. 159016–159028.
ista: 'Cyffers EAL. 2025. Setting ε is not the issue in differential privacy. 39th
Conference on Neural Information Processing Systems. NeurIPS: Neural Information
Processing Systems, Advances in Neural Information Processing Systems, vol. 38,
159016–159028.'
mla: Cyffers, Edwige Audrey Lucienne. “Setting ε Is Not the Issue in Differential
Privacy.” 39th Conference on Neural Information Processing Systems, vol.
38, Neural Information Processing Systems Foundation, 2025, pp. 159016–28, doi:10.52202/085713-4798.
short: E.A.L. Cyffers, in:, 39th Conference on Neural Information Processing Systems,
Neural Information Processing Systems Foundation, 2025, pp. 159016–159028.
conference:
end_date: 2025-12-07
location: San Diego, CA, United States
name: 'NeurIPS: Neural Information Processing Systems'
start_date: 2025-12-02
corr_author: '1'
das_tickbox: '0'
date_created: 2026-09-06T22:01:59Z
date_published: 2025-12-02T00:00:00Z
date_updated: 2026-09-10T08:44:14Z
day: '02'
department:
- _id: ChLa
doi: 10.52202/085713-4798
fulldoi: https://doi.org/10.52202/085713-4798
intvolume: ' 38'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.52202/085713-4798
month: '12'
oa: 1
oa_version: Published Version
page: 159016-159028
publication: 39th Conference on Neural Information Processing Systems
publication_identifier:
isbn:
- '9798331338275'
issn:
- 1049-5258
publication_status: published
publisher: Neural Information Processing Systems Foundation
quality_controlled: '1'
researchdata_availability: no
scopus_import: '1'
status: public
supplementarymaterial: yes
title: Setting ε is not the issue in differential privacy
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 38
year: '2025'
...
---
OA_place: publisher
OA_type: free access
_id: '22830'
abstract:
- lang: eng
text: "We introduce AutoJudge, a method that accelerates large language model (LLM)
inference with task-specific lossy speculative decoding. Instead of matching the
original model output distribution token-by-token, we identify the generated tokens
that affect the downstream quality of the response, relaxing the distribution
match guarantee so that the \"unimportant\" tokens can be generated faster. Our
approach relies on a semi‑greedy search algorithm to test which of the mismatches
between target and draft models should be corrected to preserve quality and which
ones may be skipped. We then train a lightweight classifier based on existing
LLM embeddings to predict, at inference time, which mismatching tokens can be
safely accepted without compromising the final answer quality. We evaluate AutoJudge
with multiple draft/target model pairs on mathematical reasoning and programming
benchmarks, achieving significant speedups at the cost of a minor accuracy reduction.
Notably, on GSM8K with the Llama 3.1 70B target model, our approach achieves up
to \r\n≈\r\n2\r\n×\r\n speedup \\textit{over speculative decoding} at the cost
of a \r\n≤\r\n1\r\n%\r\n drop in accuracy. When applied to the LiveCodeBench benchmark,
AutoJudge automatically detects programming-specific important tokens, accepting
\r\n≥\r\n25\r\n tokens per speculation cycle at a\r\n \r\n2\r\n%\r\n drop in Pass@1.
Our approach requires no human annotation and is easy to integrate with modern
LLM inference frameworks."
acknowledgement: "We would like to express our sincere gratitude to Denis Mazur for
his valuable contributions to the\r\nimplementation of API calls used in Algorithm
1 and for supporting inference with the Llama 405B\r\nmodel. We are also thankful
for his positive influence on the overall atmosphere and team morale\r\nthroughout
the course of this project."
alternative_title:
- Advances in Neural Information Processing Systems
article_processing_charge: No
arxiv: 1
author:
- first_name: Roman
full_name: Garipov, Roman
last_name: Garipov
- first_name: Fedor
full_name: Velikonivtsev, Fedor
last_name: Velikonivtsev
- first_name: Ivan
full_name: Ermakov, Ivan
last_name: Ermakov
- first_name: Ruslan
full_name: Svirschevski, Ruslan
last_name: Svirschevski
- first_name: Vage
full_name: Egiazarian, Vage
id: 77451e76-92b2-11ef-a4d1-8dbaa06e16ad
last_name: Egiazarian
- first_name: Max
full_name: Ryabinin, Max
last_name: Ryabinin
citation:
ama: 'Garipov R, Velikonivtsev F, Ermakov I, Svirschevski R, Egiazarian V, Ryabinin
M. AutoJudge: Judge decoding without manual annotation. In: 39th Annual Conference
on Neural Information Processing Systems. Vol 38. Neural Information Processing
Systems Foundation; 2025:104904-104941. doi:10.52202/085713-3163'
apa: 'Garipov, R., Velikonivtsev, F., Ermakov, I., Svirschevski, R., Egiazarian,
V., & Ryabinin, M. (2025). AutoJudge: Judge decoding without manual annotation.
In 39th Annual Conference on Neural Information Processing Systems (Vol.
38, pp. 104904–104941). San Diego, CA, United States: Neural Information Processing
Systems Foundation. https://doi.org/10.52202/085713-3163'
chicago: 'Garipov, Roman, Fedor Velikonivtsev, Ivan Ermakov, Ruslan Svirschevski,
Vage Egiazarian, and Max Ryabinin. “AutoJudge: Judge Decoding without Manual Annotation.”
In 39th Annual Conference on Neural Information Processing Systems, 38:104904–41.
Neural Information Processing Systems Foundation, 2025. https://doi.org/10.52202/085713-3163.'
ieee: 'R. Garipov, F. Velikonivtsev, I. Ermakov, R. Svirschevski, V. Egiazarian,
and M. Ryabinin, “AutoJudge: Judge decoding without manual annotation,” in 39th
Annual Conference on Neural Information Processing Systems, San Diego, CA,
United States, 2025, vol. 38, pp. 104904–104941.'
ista: 'Garipov R, Velikonivtsev F, Ermakov I, Svirschevski R, Egiazarian V, Ryabinin
M. 2025. AutoJudge: Judge decoding without manual annotation. 39th Annual Conference
on Neural Information Processing Systems. NeurIPS: Neural Information Processing
Systems, Advances in Neural Information Processing Systems, vol. 38, 104904–104941.'
mla: 'Garipov, Roman, et al. “AutoJudge: Judge Decoding without Manual Annotation.”
39th Annual Conference on Neural Information Processing Systems, vol. 38,
Neural Information Processing Systems Foundation, 2025, pp. 104904–41, doi:10.52202/085713-3163.'
short: R. Garipov, F. Velikonivtsev, I. Ermakov, R. Svirschevski, V. Egiazarian,
M. Ryabinin, in:, 39th Annual Conference on Neural Information Processing Systems,
Neural Information Processing Systems Foundation, 2025, pp. 104904–104941.
conference:
end_date: 2025-12-07
location: San Diego, CA, United States
name: 'NeurIPS: Neural Information Processing Systems'
start_date: 2025-12-02
das_tickbox: '0'
date_created: 2026-09-06T22:02:00Z
date_published: 2025-12-02T00:00:00Z
date_updated: 2026-09-10T11:08:33Z
day: '02'
department:
- _id: DaAl
doi: 10.52202/085713-3163
external_id:
arxiv:
- '2504.20039'
fulldoi: https://doi.org/10.52202/085713-3163
intvolume: ' 38'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.52202/085713-3163
month: '12'
oa: 1
oa_version: Published Version
page: 104904-104941
publication: 39th Annual Conference on Neural Information Processing Systems
publication_identifier:
isbn:
- '9798331338275'
issn:
- 1049-5258
publication_status: published
publisher: Neural Information Processing Systems Foundation
quality_controlled: '1'
researchdata_availability: no
scopus_import: '1'
status: public
supplementarymaterial: yes
title: 'AutoJudge: Judge decoding without manual annotation'
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 38
year: '2025'
...
---
OA_place: publisher
OA_type: free access
_id: '22828'
abstract:
- lang: eng
text: 'We propose Joint Moment Estimation (JME), a method for continually and privately
estimating both the first and second moments of a data stream with reduced noise
compared to naive approaches. JME supports the matrix mechanism and exploits a
joint sensitivity analysis to identify a privacy regime in which the second-moment
estimation incurs no additional privacy cost, thereby improving accuracy while
maintaining privacy. We demonstrate JME’s effectiveness in two applications: estimating
the running mean and covariance matrix for Gaussian density estimation and model
training with DP-Adam.'
acknowledgement: "We thank Monika Henzinger for her valuable feedback and insightful
discussions on earlier versions\r\nof this draft. We are also grateful to Mher Safaryan
for his contributions to discussions on DP-Adam.\r\nAdditionally, we thank Ryan
McKenna for suggesting Joint Clipping as a baseline.\r\nJalaj Upadhyay’s research
was funded by the Rutgers Decanal Grant no. 302918, NSF CNS 2433628,\r\nGoogle Research
Scholar Award, and Google Seed Fund Grant. A part of this work was done while\r\nvisiting
the Institute of Science and Technology Austria (ISTA).\r\nNikita Kalinin’s research
was funded in part by the Austrian Science Fund (FWF) [10.55776/COE12]"
alternative_title:
- Advances in Neural Information Processing Systems
article_processing_charge: No
author:
- first_name: Nikita
full_name: Kalinin, Nikita
id: 4b14526e-14d2-11ed-ba64-c14c9553d137
last_name: Kalinin
- first_name: Jalaj
full_name: Upadhyay, Jalaj
last_name: Upadhyay
- first_name: Christoph
full_name: Lampert, Christoph
id: 40C20FD2-F248-11E8-B48F-1D18A9856A87
last_name: Lampert
orcid: 0000-0001-8622-7887
citation:
ama: 'Kalinin N, Upadhyay J, Lampert C. Continual release moment estimation with
differential privacy. In: Advances in Neural Information Processing Systems.
Vol 38. Neural Information Processing Systems Foundation; 2025:64002-64047. doi:10.52202/085713-1924'
apa: 'Kalinin, N., Upadhyay, J., & Lampert, C. (2025). Continual release moment
estimation with differential privacy. In Advances in Neural Information Processing
Systems (Vol. 38, pp. 64002–64047). San Diego, CA, United States: Neural Information
Processing Systems Foundation. https://doi.org/10.52202/085713-1924'
chicago: Kalinin, Nikita, Jalaj Upadhyay, and Christoph Lampert. “Continual Release
Moment Estimation with Differential Privacy.” In Advances in Neural Information
Processing Systems, 38:64002–47. Neural Information Processing Systems Foundation,
2025. https://doi.org/10.52202/085713-1924.
ieee: N. Kalinin, J. Upadhyay, and C. Lampert, “Continual release moment estimation
with differential privacy,” in Advances in Neural Information Processing Systems,
San Diego, CA, United States, 2025, vol. 38, pp. 64002–64047.
ista: 'Kalinin N, Upadhyay J, Lampert C. 2025. Continual release moment estimation
with differential privacy. Advances in Neural Information Processing Systems.
NeurIPS: Neural Information Processing Systems, Advances in Neural Information
Processing Systems, vol. 38, 64002–64047.'
mla: Kalinin, Nikita, et al. “Continual Release Moment Estimation with Differential
Privacy.” Advances in Neural Information Processing Systems, vol. 38, Neural
Information Processing Systems Foundation, 2025, pp. 64002–47, doi:10.52202/085713-1924.
short: N. Kalinin, J. Upadhyay, C. Lampert, in:, Advances in Neural Information
Processing Systems, Neural Information Processing Systems Foundation, 2025, pp.
64002–64047.
conference:
location: San Diego, CA, United States
name: 'NeurIPS: Neural Information Processing Systems'
corr_author: '1'
das_tickbox: '0'
date_created: 2026-09-06T22:02:00Z
date_published: 2025-12-02T00:00:00Z
date_updated: 2026-09-10T11:17:06Z
day: '02'
department:
- _id: GradSch
- _id: ChLa
doi: 10.52202/085713-1924
fulldoi: https://doi.org/10.52202/085713-1924
intvolume: ' 38'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.52202/085713-1924
month: '12'
oa: 1
oa_version: Published Version
page: 64002-64047
publication: Advances in Neural Information Processing Systems
publication_identifier:
isbn:
- '9798331338275'
issn:
- 1049-5258
publication_status: published
publisher: Neural Information Processing Systems Foundation
quality_controlled: '1'
researchdata_availability: no
scopus_import: '1'
status: public
supplementarymaterial: no
title: Continual release moment estimation with differential privacy
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 38
year: '2025'
...