---
OA_place: publisher
OA_type: hybrid
_id: '21839'
abstract:
- lang: eng
text: "Background & Aims: To develop and validate a CT-based radiomics model to
assess HVPG and predict a composite endpoint of liver-related events (LRE: decompensation
and liver-related death) in patients with cirrhosis.\r\n\r\nMethods: This retrospective
study included 357 cirrhosis patients, who received invasive HVPG measurements,
120 liver-healthy controls (training cohort) and 85 and 100 cirrhosis patients
(internal and external validation cohorts, respectively), and contrast-enhanced
abdominal CTs. After volumetric segmentation of the liver and spleen on CT, Bayesian
parameter optimization was used for selection of extracted features and hyperparameter
tuning in random forest or elastic net models. Prediction accuracy was evaluated
using Pearson correlation coefficients of predicted (’radio-HVPG’) and invasive
HVPG. Discrimination between relevant HVPG cut-offs was determined by receiver
operating characteristic (ROC) analysis. The predictive value of radio-HVPG and
invasive-HVPG for LRE was compared using Cox regression models.\r\n\r\nResults:
Radio-HVPG, predicted by an optimized random forest model based on 74 selected
CT features, correlated with invasive-HVPG and detected clinically significant
portal hypertension (CSPH: HVPG ≥ 10 mmHg) on the internal (Pearson r = 0.63,
AUC 0.89 [95% CI: 0.81–0.96]) and external (Pearson r = 0.62, AUC 0.80 [95% CI:
0.64–0.91]) validation cohorts. Radio-HVPG predicted LRE when adjusting for MELD
and albumin (adjusted HR: 1.14 [95% CI: 1.04–1.25], p = 0.005) and performed similarly
to invasive-HVPG.\r\n\r\nConclusions: Radiomic features accurately predict HVPG
in patients with cirrhosis and allow risk stratification for LRE in a radiomics-clinical
signature."
acknowledgement: "The computational results presented were partly obtained using the
CLIP cluster (https://clip.science/). The authors thank Clemens Watzenboeck from
the Medical University of Vienna for the assistance in code upload and repository
maintenance. The authors dedicate this work to the memory of Martin Watzenboeck,
who served as first author and whose vision and scientific rigor were fundamental
to the conception and completion of this study. Open Access funding provided by
Medizinische Universitat Wien/KEMÖ. This work was supported by the Vienna Science
and Technology Fund (WWTF) through projects VRG15-005 and NXT 19-008 granted to
J.M and the Clinical Research Group MOTION, Medical University of Vienna, Vienna,
Austria – a Clinical Research Group Programme project funded by the Ludwig Boltzmann
Gesellschaft (Grant Nr LBG_KFG_22_32) with funds from the Fonds Zukunft Österreich.\r\n\r\nP-E.R.'s
research laboratory is supported by the Fondation pour la Recherche Médicale (FRM
EQU202303016287), “Institut National de la Santé et de la Recherche Médicale” (ATIP
AVENIR), the “Agence Nationale de la Recherche” (ANR-18-CE14-0006-01, RHU QUID-NASH,
ANR-18-IDEX-0001, ANR-22-CE14-0002) by « Émergence, Ville de Paris », by Fondation
ARC, by the European Union's Horizon 2020 research and innovation programme under
grant agreement No 847949 and by France 2030 RHU LIVER-TRACK."
article_number: e70633
article_processing_charge: Yes (via OA deal)
article_type: original
author:
- first_name: Celine
full_name: Sin, Celine
last_name: Sin
- first_name: Martin Luther
full_name: Watzenboeck, Martin Luther
last_name: Watzenboeck
- first_name: Eugenia B
full_name: Iofinova, Eugenia B
id: f9a17499-f6e0-11ea-865d-fdf9a3f77117
last_name: Iofinova
orcid: 0000-0002-7778-3221
- first_name: Lorenz
full_name: Balcar, Lorenz
last_name: Balcar
- first_name: Georg
full_name: Semmler, Georg
last_name: Semmler
- first_name: Bernhard
full_name: Scheiner, Bernhard
last_name: Scheiner
- first_name: Katharina
full_name: Lampichler, Katharina
last_name: Lampichler
- first_name: Mattias
full_name: Mandorfer, Mattias
last_name: Mandorfer
- first_name: Lucile
full_name: Moga, Lucile
last_name: Moga
- first_name: Pierre‐Emmanuel
full_name: Rautou, Pierre‐Emmanuel
last_name: Rautou
- first_name: Maxime
full_name: Ronot, Maxime
last_name: Ronot
- first_name: Jörg
full_name: Menche, Jörg
last_name: Menche
- first_name: Thomas
full_name: Reiberger, Thomas
last_name: Reiberger
- first_name: Martina
full_name: Scharitzer, Martina
last_name: Scharitzer
citation:
ama: Sin C, Watzenboeck ML, Iofinova EB, et al. Radiomics‐based assessment of portal
hypertension severity and risk stratification of cirrhotic patients using routine
CT scans. Liver International. 2026;46(5). doi:10.1111/liv.70633
apa: Sin, C., Watzenboeck, M. L., Iofinova, E. B., Balcar, L., Semmler, G., Scheiner,
B., … Scharitzer, M. (2026). Radiomics‐based assessment of portal hypertension
severity and risk stratification of cirrhotic patients using routine CT scans.
Liver International. Wiley. https://doi.org/10.1111/liv.70633
chicago: Sin, Celine, Martin Luther Watzenboeck, Eugenia B Iofinova, Lorenz Balcar,
Georg Semmler, Bernhard Scheiner, Katharina Lampichler, et al. “Radiomics‐based
Assessment of Portal Hypertension Severity and Risk Stratification of Cirrhotic
Patients Using Routine CT Scans.” Liver International. Wiley, 2026. https://doi.org/10.1111/liv.70633.
ieee: C. Sin et al., “Radiomics‐based assessment of portal hypertension severity
and risk stratification of cirrhotic patients using routine CT scans,” Liver
International, vol. 46, no. 5. Wiley, 2026.
ista: Sin C, Watzenboeck ML, Iofinova EB, Balcar L, Semmler G, Scheiner B, Lampichler
K, Mandorfer M, Moga L, Rautou P, Ronot M, Menche J, Reiberger T, Scharitzer M.
2026. Radiomics‐based assessment of portal hypertension severity and risk stratification
of cirrhotic patients using routine CT scans. Liver International. 46(5), e70633.
mla: Sin, Celine, et al. “Radiomics‐based Assessment of Portal Hypertension Severity
and Risk Stratification of Cirrhotic Patients Using Routine CT Scans.” Liver
International, vol. 46, no. 5, e70633, Wiley, 2026, doi:10.1111/liv.70633.
short: C. Sin, M.L. Watzenboeck, E.B. Iofinova, L. Balcar, G. Semmler, B. Scheiner,
K. Lampichler, M. Mandorfer, L. Moga, P. Rautou, M. Ronot, J. Menche, T. Reiberger,
M. Scharitzer, Liver International 46 (2026).
date_created: 2026-05-07T08:51:47Z
date_published: 2026-05-01T00:00:00Z
date_updated: 2026-05-18T07:20:20Z
day: '01'
ddc:
- '570'
doi: 10.1111/liv.70633
external_id:
pmid:
- '41943460'
file:
- access_level: open_access
checksum: fafcc0b88b8e8caed85849627305d9ba
content_type: application/pdf
creator: dernst
date_created: 2026-05-18T07:10:31Z
date_updated: 2026-05-18T07:10:31Z
file_id: '21888'
file_name: 2026_LiverInternational_Sin.pdf
file_size: 3550462
relation: main_file
success: 1
file_date_updated: 2026-05-18T07:10:31Z
has_accepted_license: '1'
intvolume: ' 46'
issue: '5'
keyword:
- computed tomography
- liver
- portal hypertension
- radiomics
- spleen
language:
- iso: eng
license: https://creativecommons.org/licenses/by-nc-nd/4.0/
month: '05'
oa: 1
oa_version: Published Version
pmid: 1
publication: Liver International
publication_identifier:
eissn:
- 1478-3231
issn:
- 1478-3223
publication_status: published
publisher: Wiley
quality_controlled: '1'
scopus_import: '1'
status: public
title: Radiomics‐based assessment of portal hypertension severity and risk stratification
of cirrhotic patients using routine CT scans
tmp:
image: /images/cc_by_nc_nd.png
legal_code_url: https://creativecommons.org/licenses/by-nc-nd/4.0/legalcode
name: Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International
(CC BY-NC-ND 4.0)
short: CC BY-NC-ND (4.0)
type: journal_article
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 46
year: '2026'
...
---
OA_place: publisher
_id: '21854'
abstract:
- lang: eng
text: "As neural-network-based models grow both in size and popularity, interest
has grown in making the models smaller and more efficient to train. To that end,
many methods have been proposed to prune models by reducing their number of nonzero
parameters. Additionally, parameter-efficient fine-tuning, in which a much smaller
number of parameters than the total contained in the model is updated during training,
has become very popular, especially in the space of Large Language Models. At
the same time, the increasingly routine deployment of machine learning in real-world
applications has spurred a drive to make them more trustworthy - in the sense
of, among other things, being unbiased, interpretable, and editable. In this thesis,
we examine the interplay between efficiency and trustworthiness.\r\n\r\nFirst,
we analyze the effects of model pruning on bias in computer vision models, demonstrating
that increased sparsity leads to greater bias, largely as a function of increased
model uncertainty in marginal cases. Based on this observation, we propose several
bias mitigation techniques. Then, we demonstrate that example-specific model pruning
can improve model interpretation methods while improving pruning efficiency to
make example-specific model pruning feasible in real time. Then, we investigate
the effectiveness of parameter-efficient and data-efficient model personalization
via fine-tuning, demonstrating that it is highly feasible with very small computational
and data resources. Finally, we consider efficiency in editing model knowledge
using a custom synthetic data framework, demonstrating that parameter-efficient,
low-rank fine-tuning frequently outperforms full-rank fine-tuning, and, additionally,
that restricting which model blocks are fine-tuned frequently improves results.
Together, the results in this thesis provide new insights and techniques for combining
trustworthiness and efficiency during neural network inference and training.\r\n\r\n-----------------“In
reference to IEEE copyrighted material which is used with permission in this thesis,
the IEEE does not endorse any of [name of university or educational entity]’s
products or services. Internal or personal use of this material is permitted.
If interested in reprinting/republishing IEEE copyrighted material for advertising
or promotional purposes or for creating new collective works for resale or redistribution,
please go to http://www.ieee.org/publications_standards/publications/rights/rights_link.html
to learn how to obtain a License from RightsLink. If applicable, University Microfilms
and/or ProQuest Library, or the Archives of Canada may supply single copies of
the dissertation.”"
acknowledged_ssus:
- _id: ScienComp
acknowledgement: "The research in this Ph.D. was funded in whole\r\nor in part by
the Austrian Science Fund (FWF) W1260-N35 (Vienna Graduate School for\r\nComputational
Optimization). For open access purposes the author has applied a CC BY\r\npublic
copyright license to any author accepted manuscript version arising from this submission\r\nwherever
possible. Additionally, I am grateful to Alois Schlögl, Waleed Khalid, and the rest
of\r\nthe ISTA Scientific Computing team for building and maintaining the infrastructure
I used\r\nto run experiments. I’m also deeply grateful to the Alistarh group’s administrative
assistant,\r\nChristine Francois, who always deals with our nonsense with common
sense and a smile.\r\n"
alternative_title:
- ISTA Thesis
article_processing_charge: No
author:
- first_name: Eugenia B
full_name: Iofinova, Eugenia B
id: f9a17499-f6e0-11ea-865d-fdf9a3f77117
last_name: Iofinova
orcid: 0000-0002-7778-3221
citation:
ama: Iofinova EB. On the utility and effects of efficiency in artificial neural
networks. 2026. doi:10.15479/AT-ISTA-21854
apa: Iofinova, E. B. (2026). On the utility and effects of efficiency in artificial
neural networks. Institute of Science and Technology Austria. https://doi.org/10.15479/AT-ISTA-21854
chicago: Iofinova, Eugenia B. “On the Utility and Effects of Efficiency in Artificial
Neural Networks.” Institute of Science and Technology Austria, 2026. https://doi.org/10.15479/AT-ISTA-21854.
ieee: E. B. Iofinova, “On the utility and effects of efficiency in artificial neural
networks,” Institute of Science and Technology Austria, 2026.
ista: Iofinova EB. 2026. On the utility and effects of efficiency in artificial
neural networks. Institute of Science and Technology Austria.
mla: Iofinova, Eugenia B. On the Utility and Effects of Efficiency in Artificial
Neural Networks. Institute of Science and Technology Austria, 2026, doi:10.15479/AT-ISTA-21854.
short: E.B. Iofinova, On the Utility and Effects of Efficiency in Artificial Neural
Networks, Institute of Science and Technology Austria, 2026.
corr_author: '1'
date_created: 2026-05-11T08:43:22Z
date_published: 2026-05-11T00:00:00Z
date_updated: 2026-05-19T11:20:28Z
day: '11'
ddc:
- '000'
degree_awarded: PhD
department:
- _id: GradSch
- _id: DaAl
doi: 10.15479/AT-ISTA-21854
file:
- access_level: closed
checksum: 2e148dad920e3f9b7c32796e0ba2e5f7
content_type: application/zip
creator: eiofinov
date_created: 2026-05-11T08:36:01Z
date_updated: 2026-05-11T08:36:01Z
file_id: '21856'
file_name: EIofinova_thesis_FinalVersion.zip
file_size: 28479571
relation: source_file
- access_level: open_access
checksum: b10c2933f386f532b2dbf28b19c5525c
content_type: application/pdf
creator: eiofinov
date_created: 2026-05-13T13:10:48Z
date_updated: 2026-05-13T13:10:48Z
file_id: '21877'
file_name: 2026_Iofinova_Eugenia_Thesis.pdf
file_size: 18137757
relation: main_file
success: 1
file_date_updated: 2026-05-13T13:10:48Z
has_accepted_license: '1'
language:
- iso: eng
month: '05'
oa: 1
oa_version: Published Version
page: '237'
project:
- _id: 9B9290DE-BA93-11EA-9121-9846C619BF3A
grant_number: W1260-N35
name: Vienna Graduate School on Computational Optimization
publication_identifier:
issn:
- 2663-337X
publication_status: published
publisher: Institute of Science and Technology Austria
related_material:
record:
- id: '14771'
relation: part_of_dissertation
status: public
- id: '18121'
relation: part_of_dissertation
status: public
- id: '21858'
relation: part_of_dissertation
status: public
- id: '21859'
relation: part_of_dissertation
status: public
- id: '21857'
relation: part_of_dissertation
status: public
status: public
supervisor:
- first_name: Dan-Adrian
full_name: Alistarh, Dan-Adrian
id: 4A899BFC-F248-11E8-B48F-1D18A9856A87
last_name: Alistarh
orcid: 0000-0003-3650-940X
title: On the utility and effects of efficiency in artificial neural networks
type: dissertation
user_id: 8b945eb4-e2f2-11eb-945a-df72226e66a9
year: '2026'
...
---
OA_place: publisher
OA_type: green
_id: '21857'
abstract:
- lang: eng
text: "The availability of powerful open-source large language models (LLMs) opens
exciting use cases, such as using personal data to fine-tune these models to imitate
a user’s unique writing style. Two key requirements for this functionality are
personalization–in the sense that the output should recognizably reflect the user’s
own writing style—and privacy–users may justifiably be wary of uploading extremely
personal data, such as their email archive, to a third-party service. In this
paper, we demonstrate the feasibility of training and running such an assistant,
which we call Panza, on commodity hardware, for the specific use case of email
generation. Panza’s personalization features are based on a combination of parameter-efficient
fine-tuning using a variant of the Reverse Instructions technique [1] and Retrieval-Augmented
Generation (RAG) [2]. We demonstrate that this combination allows us to fine-tune
an LLM to reflect a user’s writing style using limited data, while executing on
extremely limited resources, e.g. on a free Google Colab instance. Our key methodological
contribution is the first detailed study of evaluation metrics for this task,
and\r\nof how different choices of system components–the use of RAG and of different
fine-tuning approaches–impact the system’s performance. Additionally, we demonstrate
that very little data - under 100 email samples - are sufficient to create models
that convincingly imitate humans, showcasing a previously unknown attack vector
in language models. We are releasing the full Panza code as well as three new
email datasets licensed for research use."
article_number: '81'
article_processing_charge: No
author:
- first_name: Armand
full_name: Nicolicioiu, Armand
last_name: Nicolicioiu
- first_name: Eugenia B
full_name: Iofinova, Eugenia B
id: f9a17499-f6e0-11ea-865d-fdf9a3f77117
last_name: Iofinova
orcid: 0000-0002-7778-3221
- first_name: Andrej
full_name: Jovanovic, Andrej
last_name: Jovanovic
- first_name: Eldar
full_name: Kurtic, Eldar
id: 47beb3a5-07b5-11eb-9b87-b108ec578218
last_name: Kurtic
- first_name: Mahdi
full_name: Nikdan, Mahdi
id: 66374281-f394-11eb-9cf6-869147deecc0
last_name: Nikdan
- first_name: Andrei
full_name: Panferov, Andrei
id: 2c18daae-4dbe-11ef-8491-98ce2d960f09
last_name: Panferov
- first_name: Ilia
full_name: Markov, Ilia
id: D0CF4148-C985-11E9-8066-0BDEE5697425
last_name: Markov
- first_name: Nir
full_name: Shavit, Nir
last_name: Shavit
- first_name: Dan-Adrian
full_name: Alistarh, Dan-Adrian
id: 4A899BFC-F248-11E8-B48F-1D18A9856A87
last_name: Alistarh
orcid: 0000-0003-3650-940X
citation:
ama: 'Nicolicioiu A, Iofinova EB, Jovanovic A, et al. Panza: Investigating the
Feasibility of Fully-Local Personalized Text Generation. OpenReview; 2026.'
apa: 'Nicolicioiu, A., Iofinova, E. B., Jovanovic, A., Kurtic, E., Nikdan, M., Panferov,
A., … Alistarh, D.-A. (2026). Panza: Investigating the feasibility of fully-local
personalized text generation. Third Conference on Parsimony and Learning
(Proceedings Track). Tübíngen, Germany: OpenReview.'
chicago: 'Nicolicioiu, Armand, Eugenia B Iofinova, Andrej Jovanovic, Eldar Kurtic,
Mahdi Nikdan, Andrei Panferov, Ilia Markov, Nir Shavit, and Dan-Adrian Alistarh.
Panza: Investigating the Feasibility of Fully-Local Personalized Text Generation.
Third Conference on Parsimony and Learning (Proceedings Track). OpenReview,
2026.'
ieee: 'A. Nicolicioiu et al., Panza: Investigating the feasibility of
fully-local personalized text generation. OpenReview, 2026.'
ista: 'Nicolicioiu A, Iofinova EB, Jovanovic A, Kurtic E, Nikdan M, Panferov A,
Markov I, Shavit N, Alistarh D-A. 2026. Panza: Investigating the feasibility of
fully-local personalized text generation, OpenReview,p.'
mla: 'Nicolicioiu, Armand, et al. “Panza: Investigating the Feasibility of Fully-Local
Personalized Text Generation.” Third Conference on Parsimony and Learning (Proceedings
Track), 81, OpenReview, 2026.'
short: 'A. Nicolicioiu, E.B. Iofinova, A. Jovanovic, E. Kurtic, M. Nikdan, A. Panferov,
I. Markov, N. Shavit, D.-A. Alistarh, Panza: Investigating the Feasibility of
Fully-Local Personalized Text Generation, OpenReview, 2026.'
conference:
end_date: 2026-03-26
location: Tübíngen, Germany
name: 'CPAL: Conference on Parsimony and Learning'
start_date: 2026-03-23
corr_author: '1'
date_created: 2026-05-11T08:50:28Z
date_published: 2026-03-06T00:00:00Z
date_updated: 2026-05-19T11:20:27Z
day: '06'
department:
- _id: GradSch
- _id: DaAl
keyword:
- LLMs
- PEFT
- LoRA
- personalization
- efficient ML
language:
- iso: eng
license: https://creativecommons.org/licenses/by/4.0/
main_file_link:
- open_access: '1'
url: https://openreview.net/pdf?id=soFWnTqd23
month: '03'
oa: 1
oa_version: Accepted Version
publication: Third Conference on Parsimony and Learning (Proceedings Track)
publication_status: published
publisher: OpenReview
quality_controlled: '1'
related_material:
record:
- id: '21854'
relation: dissertation_contains
status: public
status: public
title: 'Panza: Investigating the feasibility of fully-local personalized text generation'
tmp:
image: /images/cc_by.png
legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode
name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)
short: CC BY (4.0)
type: conference_poster
user_id: 8b945eb4-e2f2-11eb-945a-df72226e66a9
year: '2026'
...
---
OA_place: repository
OA_type: green
_id: '21859'
abstract:
- lang: eng
text: As artificial neural networks, and specifically large language models, have
improved rapidly in capabilities and quality, they have increasingly been deployed
in real-world applications, from customer service to Google search, despite the
fact that they frequently make factually incorrect or undesirable statements.
This trend has inspired practical and academic interest in model editing, that
is, in adjusting the weights of the model to modify its likely outputs for queries
relating to a specific fact or set of facts. This may be done either to amend
a fact or set of facts, for instance, to fix a frequent error in the training
data, or to suppress a fact or set of facts entirely, for instance, in case of
dangerous knowledge. Multiple methods have been proposed to do such edits. However,
at the same time, it has been shown that such model editing can be brittle and
incomplete. Moreover the effectiveness of any model editing method necessarily
depends on the data on which the model is trained, and, therefore, a good understanding
of the interaction of the training data distribution and the way it is stored
in the network is necessary and helpful to reliably perform model editing. However,
working with large language models trained on real-world data does not allow us
to understand this relationship or fully measure the effects of model editing.
We therefore propose Behemoth, a fully synthetic data generation framework. To
demonstrate the practical insights from the framework, we explore model editing
in the context of simple tabular data, demonstrating surprising findings that,
in some cases, echo real-world results, for instance, that in some cases restricting
the update rank results in a more effective update.
acknowledged_ssus:
- _id: ScienComp
acknowledgement: "EI thanks Weiwei Yang, Janardhan Kulkani, and Kate Lytvynets for
their advice and support in\r\ndeveloping an earlier version of the Behemoth library.
This research was supported by the Scientific\r\nService Units (SSU) of IST Austria
through resources provided by Scientific Computing (SciComp).\r\nEI was supported
in part by the FWF DK VGSCO, grant agreement number W1260-N35.\r\n"
article_processing_charge: No
arxiv: 1
author:
- first_name: Eugenia B
full_name: Iofinova, Eugenia B
id: f9a17499-f6e0-11ea-865d-fdf9a3f77117
last_name: Iofinova
orcid: 0000-0002-7778-3221
- first_name: Dan-Adrian
full_name: Alistarh, Dan-Adrian
id: 4A899BFC-F248-11E8-B48F-1D18A9856A87
last_name: Alistarh
orcid: 0000-0003-3650-940X
citation:
ama: 'Iofinova EB, Alistarh D-A. Behemoth: Benchmarking unlearning in LLMs using
fully synthetic data. arXiv. doi:10.48550/arXiv.2601.23153'
apa: 'Iofinova, E. B., & Alistarh, D.-A. (n.d.). Behemoth: Benchmarking unlearning
in LLMs using fully synthetic data. arXiv. https://doi.org/10.48550/arXiv.2601.23153'
chicago: 'Iofinova, Eugenia B, and Dan-Adrian Alistarh. “Behemoth: Benchmarking
Unlearning in LLMs Using Fully Synthetic Data.” ArXiv, n.d. https://doi.org/10.48550/arXiv.2601.23153.'
ieee: 'E. B. Iofinova and D.-A. Alistarh, “Behemoth: Benchmarking unlearning in
LLMs using fully synthetic data,” arXiv. .'
ista: 'Iofinova EB, Alistarh D-A. Behemoth: Benchmarking unlearning in LLMs using
fully synthetic data. arXiv, 10.48550/arXiv.2601.23153.'
mla: 'Iofinova, Eugenia B., and Dan-Adrian Alistarh. “Behemoth: Benchmarking Unlearning
in LLMs Using Fully Synthetic Data.” ArXiv, doi:10.48550/arXiv.2601.23153.'
short: E.B. Iofinova, D.-A. Alistarh, ArXiv (n.d.).
corr_author: '1'
date_created: 2026-05-11T08:58:07Z
date_published: 2026-01-30T00:00:00Z
date_updated: 2026-05-19T11:20:27Z
day: '30'
department:
- _id: GradSch
- _id: DaAl
doi: 10.48550/arXiv.2601.23153
external_id:
arxiv:
- '2601.23153'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2601.23153
month: '01'
oa: 1
oa_version: Preprint
project:
- _id: 9B9290DE-BA93-11EA-9121-9846C619BF3A
grant_number: W1260-N35
name: Vienna Graduate School on Computational Optimization
publication: arXiv
publication_status: draft
related_material:
record:
- id: '21854'
relation: dissertation_contains
status: public
status: public
title: 'Behemoth: Benchmarking unlearning in LLMs using fully synthetic data'
type: preprint
user_id: 8b945eb4-e2f2-11eb-945a-df72226e66a9
year: '2026'
...
---
OA_place: repository
OA_type: green
_id: '21858'
abstract:
- lang: eng
text: "The recent surge in high-quality open-source Generative AI text models (colloquially:
LLMs), as well as efficient finetuning techniques, have opened the possibility
of creating high-quality personalized models that generate text attuned to a specific
individual’s needs and are capable of credibly imitating their writing style by
refining an open-source model with that person’s own data. The technology to create
such models is accessible to private individuals, and training and running such
models can be done cheaply on consumer-grade hardware. While these advancements
are a huge gain for usability and privacy, this position paper argues that the
practical feasibility of impersonating specific individuals also introduces novel
safety risks. For instance, this technology enables the creation of phishing emails\r\nor
fraudulent social media accounts, based on small amounts of publicly available
text, or by the individuals themselves to escape AI text detection. We further
argue that these risks are complementary to—and distinct from—the much-discussed
risks of other impersonation attacks such as image, voice, or video deepfakes,
and are not adequately addressed by the larger research community, or the current
generation of open- and closed-source models."
acknowledgement: "This research was supported by the Scientific Service Units (SSU)
of IST Austria through resources\r\nprovided by Scientific Computing (SciComp).
EI was supported in part by the FWF DK VGSCO,\r\ngrant agreement number W1260-N35.
AJ was supported in part by ERC Proof-of-Concept Grant\r\nFastML, grant agreement
101158077."
article_processing_charge: No
arxiv: 1
author:
- first_name: Eugenia B
full_name: Iofinova, Eugenia B
id: f9a17499-f6e0-11ea-865d-fdf9a3f77117
last_name: Iofinova
orcid: 0000-0002-7778-3221
- first_name: Andrej
full_name: Jovanovic, Andrej
last_name: Jovanovic
- first_name: Dan-Adrian
full_name: Alistarh, Dan-Adrian
id: 4A899BFC-F248-11E8-B48F-1D18A9856A87
last_name: Alistarh
orcid: 0000-0003-3650-940X
citation:
ama: 'Iofinova EB, Jovanovic A, Alistarh D-A. Position: It’s time to act on the
risk of efficient personalized text generation. arXiv. doi:10.48550/arXiv.2502.06560'
apa: 'Iofinova, E. B., Jovanovic, A., & Alistarh, D.-A. (n.d.). Position: It’s
time to act on the risk of efficient personalized text generation. arXiv.
https://doi.org/10.48550/arXiv.2502.06560'
chicago: 'Iofinova, Eugenia B, Andrej Jovanovic, and Dan-Adrian Alistarh. “Position:
It’s Time to Act on the Risk of Efficient Personalized Text Generation.” ArXiv,
n.d. https://doi.org/10.48550/arXiv.2502.06560.'
ieee: 'E. B. Iofinova, A. Jovanovic, and D.-A. Alistarh, “Position: It’s time to
act on the risk of efficient personalized text generation,” arXiv. .'
ista: 'Iofinova EB, Jovanovic A, Alistarh D-A. Position: It’s time to act on the
risk of efficient personalized text generation. arXiv, 10.48550/arXiv.2502.06560.'
mla: 'Iofinova, Eugenia B., et al. “Position: It’s Time to Act on the Risk of Efficient
Personalized Text Generation.” ArXiv, doi:10.48550/arXiv.2502.06560.'
short: E.B. Iofinova, A. Jovanovic, D.-A. Alistarh, ArXiv (n.d.).
corr_author: '1'
date_created: 2026-05-11T08:55:23Z
date_published: 2025-06-02T00:00:00Z
date_updated: 2026-05-19T11:20:27Z
day: '02'
department:
- _id: GradSch
- _id: DaAl
doi: 10.48550/arXiv.2502.06560
external_id:
arxiv:
- '2502.06560'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2502.06560
month: '06'
oa: 1
oa_version: Preprint
project:
- _id: 8e35c14b-16d5-11f0-9cad-a3fc35339161
grant_number: '101158077'
name: 'FastML: Efficient and Cost-Effective Distributed Machine Learning'
- _id: 9B9290DE-BA93-11EA-9121-9846C619BF3A
grant_number: W1260-N35
name: Vienna Graduate School on Computational Optimization
publication: arXiv
publication_status: draft
related_material:
record:
- id: '21854'
relation: dissertation_contains
status: public
status: public
title: 'Position: It''s time to act on the risk of efficient personalized text generation'
type: preprint
user_id: 8b945eb4-e2f2-11eb-945a-df72226e66a9
year: '2025'
...
---
_id: '18121'
abstract:
- lang: eng
text: It is known that sparsity can improve interpretability for deep neural networks.
However, existing methods in the area either require networks that are pre-trained
with sparsity constraints, or impose sparsity after the fact, altering the network’s
general behavior. In this paper, we demonstrate, for the first time, that sparsity
can instead be incorporated into the interpretation process itself, as a sample-specific
preprocessing step. Unlike previous work, this approach, which we call SPADE,
does not place constraints on the trained model and does not affect its behavior
during inference on the sample. Given a trained model and a target sample, SPADE
uses sample-targeted pruning to provide a "trace" of the network’s execution on
the sample, reducing the network to the most important connections prior to computing
an interpretation. We demonstrate that preprocessing with SPADE significantly
increases the accuracy of image saliency maps across several interpretability
methods. Additionally, SPADE improves the usefulness of neuron visualizations,
aiding humans in reasoning about network behavior. Our code is available at https://github.com/IST-DASLab/SPADE.
acknowledged_ssus:
- _id: ScienComp
acknowledgement: The authors would like to thank Stephen Casper and Tony Wang for
their feedback on this work, and Eldar Kurtic for his advice on aspects of the project.
This research was supported by the Scientific Service Units (SSU) of IST Austria
through resources provided by Scientific Computing (SciComp). EI was supported in
part by the FWF DK VGSCO, grant agreement number W1260-N35.
alternative_title:
- PMLR
article_processing_charge: No
arxiv: 1
author:
- first_name: Arshia Soltani
full_name: Moakhar, Arshia Soltani
last_name: Moakhar
- first_name: Eugenia B
full_name: Iofinova, Eugenia B
id: f9a17499-f6e0-11ea-865d-fdf9a3f77117
last_name: Iofinova
orcid: 0000-0002-7778-3221
- first_name: Elias
full_name: Frantar, Elias
id: 09a8f98d-ec99-11ea-ae11-c063a7b7fe5f
last_name: Frantar
- first_name: Dan-Adrian
full_name: Alistarh, Dan-Adrian
id: 4A899BFC-F248-11E8-B48F-1D18A9856A87
last_name: Alistarh
orcid: 0000-0003-3650-940X
citation:
ama: 'Moakhar AS, Iofinova EB, Frantar E, Alistarh D-A. SPADE: Sparsity-guided debugging
for deep neural networks. In: Proceedings of the 41st International Conference
on Machine Learning. Vol 235. ML Research Press; 2024:45955-45987.'
apa: 'Moakhar, A. S., Iofinova, E. B., Frantar, E., & Alistarh, D.-A. (2024).
SPADE: Sparsity-guided debugging for deep neural networks. In Proceedings of
the 41st International Conference on Machine Learning (Vol. 235, pp. 45955–45987).
Vienna, Austria: ML Research Press.'
chicago: 'Moakhar, Arshia Soltani, Eugenia B Iofinova, Elias Frantar, and Dan-Adrian
Alistarh. “SPADE: Sparsity-Guided Debugging for Deep Neural Networks.” In Proceedings
of the 41st International Conference on Machine Learning, 235:45955–87. ML
Research Press, 2024.'
ieee: 'A. S. Moakhar, E. B. Iofinova, E. Frantar, and D.-A. Alistarh, “SPADE: Sparsity-guided
debugging for deep neural networks,” in Proceedings of the 41st International
Conference on Machine Learning, Vienna, Austria, 2024, vol. 235, pp. 45955–45987.'
ista: 'Moakhar AS, Iofinova EB, Frantar E, Alistarh D-A. 2024. SPADE: Sparsity-guided
debugging for deep neural networks. Proceedings of the 41st International Conference
on Machine Learning. ICML: International Conference on Machine Learning, PMLR,
vol. 235, 45955–45987.'
mla: 'Moakhar, Arshia Soltani, et al. “SPADE: Sparsity-Guided Debugging for Deep
Neural Networks.” Proceedings of the 41st International Conference on Machine
Learning, vol. 235, ML Research Press, 2024, pp. 45955–87.'
short: A.S. Moakhar, E.B. Iofinova, E. Frantar, D.-A. Alistarh, in:, Proceedings
of the 41st International Conference on Machine Learning, ML Research Press, 2024,
pp. 45955–45987.
conference:
end_date: 2024-07-27
location: Vienna, Austria
name: 'ICML: International Conference on Machine Learning'
start_date: 2024-07-21
corr_author: '1'
date_created: 2024-09-22T22:01:46Z
date_published: 2024-09-01T00:00:00Z
date_updated: 2026-05-19T11:20:27Z
day: '01'
department:
- _id: DaAl
external_id:
arxiv:
- '2310.04519'
intvolume: ' 235'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2310.04519
month: '09'
oa: 1
oa_version: Preprint
page: 45955-45987
project:
- _id: 9B9290DE-BA93-11EA-9121-9846C619BF3A
grant_number: W1260-N35
name: Vienna Graduate School on Computational Optimization
publication: Proceedings of the 41st International Conference on Machine Learning
publication_identifier:
eissn:
- 2640-3498
publication_status: published
publisher: ML Research Press
quality_controlled: '1'
related_material:
link:
- relation: software
url: https://github.com/IST-DASLab/SPADE
record:
- id: '21854'
relation: dissertation_contains
status: public
scopus_import: '1'
status: public
title: 'SPADE: Sparsity-guided debugging for deep neural networks'
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 235
year: '2024'
...
---
_id: '14460'
abstract:
- lang: eng
text: We provide an efficient implementation of the backpropagation algorithm, specialized
to the case where the weights of the neural network being trained are sparse.
Our algorithm is general, as it applies to arbitrary (unstructured) sparsity and
common layer types (e.g., convolutional or linear). We provide a fast vectorized
implementation on commodity CPUs, and show that it can yield speedups in end-to-end
runtime experiments, both in transfer learning using already-sparsified networks,
and in training sparse networks from scratch. Thus, our results provide the first
support for sparse training on commodity hardware.
acknowledgement: 'We would like to thank Elias Frantar for his valuable assistance
and support at the outset of this project, and the anonymous ICML and SNN reviewers
for very constructive feedback. EI was supported in part by the FWF DK VGSCO, grant
agreement number W1260-N35. DA acknowledges generous ERC support, via Starting Grant
805223 ScaleML. '
alternative_title:
- PMLR
article_processing_charge: No
arxiv: 1
author:
- first_name: Mahdi
full_name: Nikdan, Mahdi
id: 66374281-f394-11eb-9cf6-869147deecc0
last_name: Nikdan
- first_name: Tommaso
full_name: Pegolotti, Tommaso
last_name: Pegolotti
- first_name: Eugenia B
full_name: Iofinova, Eugenia B
id: f9a17499-f6e0-11ea-865d-fdf9a3f77117
last_name: Iofinova
orcid: 0000-0002-7778-3221
- first_name: Eldar
full_name: Kurtic, Eldar
id: 47beb3a5-07b5-11eb-9b87-b108ec578218
last_name: Kurtic
- first_name: Dan-Adrian
full_name: Alistarh, Dan-Adrian
id: 4A899BFC-F248-11E8-B48F-1D18A9856A87
last_name: Alistarh
orcid: 0000-0003-3650-940X
citation:
ama: 'Nikdan M, Pegolotti T, Iofinova EB, Kurtic E, Alistarh D-A. SparseProp: Efficient
sparse backpropagation for faster training of neural networks at the edge. In:
Proceedings of the 40th International Conference on Machine Learning. Vol
202. ML Research Press; 2023:26215-26227.'
apa: 'Nikdan, M., Pegolotti, T., Iofinova, E. B., Kurtic, E., & Alistarh, D.-A.
(2023). SparseProp: Efficient sparse backpropagation for faster training of neural
networks at the edge. In Proceedings of the 40th International Conference on
Machine Learning (Vol. 202, pp. 26215–26227). Honolulu, Hawaii, HI, United
States: ML Research Press.'
chicago: 'Nikdan, Mahdi, Tommaso Pegolotti, Eugenia B Iofinova, Eldar Kurtic, and
Dan-Adrian Alistarh. “SparseProp: Efficient Sparse Backpropagation for Faster
Training of Neural Networks at the Edge.” In Proceedings of the 40th International
Conference on Machine Learning, 202:26215–27. ML Research Press, 2023.'
ieee: 'M. Nikdan, T. Pegolotti, E. B. Iofinova, E. Kurtic, and D.-A. Alistarh, “SparseProp:
Efficient sparse backpropagation for faster training of neural networks at the
edge,” in Proceedings of the 40th International Conference on Machine Learning,
Honolulu, Hawaii, HI, United States, 2023, vol. 202, pp. 26215–26227.'
ista: 'Nikdan M, Pegolotti T, Iofinova EB, Kurtic E, Alistarh D-A. 2023. SparseProp:
Efficient sparse backpropagation for faster training of neural networks at the
edge. Proceedings of the 40th International Conference on Machine Learning. ICML:
International Conference on Machine Learning, PMLR, vol. 202, 26215–26227.'
mla: 'Nikdan, Mahdi, et al. “SparseProp: Efficient Sparse Backpropagation for Faster
Training of Neural Networks at the Edge.” Proceedings of the 40th International
Conference on Machine Learning, vol. 202, ML Research Press, 2023, pp. 26215–27.'
short: M. Nikdan, T. Pegolotti, E.B. Iofinova, E. Kurtic, D.-A. Alistarh, in:, Proceedings
of the 40th International Conference on Machine Learning, ML Research Press, 2023,
pp. 26215–26227.
conference:
end_date: 2023-07-29
location: Honolulu, Hawaii, HI, United States
name: 'ICML: International Conference on Machine Learning'
start_date: 2023-07-23
corr_author: '1'
date_created: 2023-10-29T23:01:17Z
date_published: 2023-07-30T00:00:00Z
date_updated: 2025-04-14T07:49:12Z
day: '30'
department:
- _id: DaAl
ec_funded: 1
external_id:
arxiv:
- '2302.04852'
intvolume: ' 202'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2302.04852
month: '07'
oa: 1
oa_version: Preprint
page: 26215-26227
project:
- _id: 268A44D6-B435-11E9-9278-68D0E5697425
call_identifier: H2020
grant_number: '805223'
name: Elastic Coordination for Scalable Machine Learning
publication: Proceedings of the 40th International Conference on Machine Learning
publication_identifier:
eissn:
- 2640-3498
publication_status: published
publisher: ML Research Press
quality_controlled: '1'
scopus_import: '1'
status: public
title: 'SparseProp: Efficient sparse backpropagation for faster training of neural
networks at the edge'
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 202
year: '2023'
...
---
_id: '14771'
abstract:
- lang: eng
text: Pruning—that is, setting a significant subset of the parameters of a neural
network to zero—is one of the most popular methods of model compression. Yet,
several recent works have raised the issue that pruning may induce or exacerbate
bias in the output of the compressed model. Despite existing evidence for this
phenomenon, the relationship between neural network pruning and induced bias is
not well-understood. In this work, we systematically investigate and characterize
this phenomenon in Convolutional Neural Networks for computer vision. First, we
show that it is in fact possible to obtain highly-sparse models, e.g. with less
than 10% remaining weights, which do not decrease in accuracy nor substantially
increase in bias when compared to dense models. At the same time, we also find
that, at higher sparsities, pruned models exhibit higher uncertainty in their
outputs, as well as increased correlations, which we directly link to increased
bias. We propose easy-to-use criteria which, based only on the uncompressed model,
establish whether bias will increase with pruning, and identify the samples most
susceptible to biased predictions post-compression. Our code can be found at https://github.com/IST-DASLab/pruned-vision-model-bias.
acknowledgement: The authors would like to sincerely thank Sara Hooker for her feedback
during the development of this work. EI was supported in part by the FWF DK VGSCO,
grant agreement number W1260-N35. AP and DA acknowledge generous ERC support, via
Starting Grant 805223 ScaleML.
article_processing_charge: No
arxiv: 1
author:
- first_name: Eugenia B
full_name: Iofinova, Eugenia B
id: f9a17499-f6e0-11ea-865d-fdf9a3f77117
last_name: Iofinova
orcid: 0000-0002-7778-3221
- first_name: Elena-Alexandra
full_name: Peste, Elena-Alexandra
id: 32D78294-F248-11E8-B48F-1D18A9856A87
last_name: Peste
- first_name: Dan-Adrian
full_name: Alistarh, Dan-Adrian
id: 4A899BFC-F248-11E8-B48F-1D18A9856A87
last_name: Alistarh
orcid: 0000-0003-3650-940X
citation:
ama: 'Iofinova EB, Krumes A, Alistarh D-A. Bias in pruned vision models: In-depth
analysis and countermeasures. In: 2023 IEEE/CVF Conference on Computer Vision
and Pattern Recognition. IEEE; 2023:24364-24373. doi:10.1109/cvpr52729.2023.02334'
apa: 'Iofinova, E. B., Krumes, A., & Alistarh, D.-A. (2023). Bias in pruned
vision models: In-depth analysis and countermeasures. In 2023 IEEE/CVF Conference
on Computer Vision and Pattern Recognition (pp. 24364–24373). Vancouver, BC,
Canada: IEEE. https://doi.org/10.1109/cvpr52729.2023.02334'
chicago: 'Iofinova, Eugenia B, Alexandra Krumes, and Dan-Adrian Alistarh. “Bias
in Pruned Vision Models: In-Depth Analysis and Countermeasures.” In 2023 IEEE/CVF
Conference on Computer Vision and Pattern Recognition, 24364–73. IEEE, 2023.
https://doi.org/10.1109/cvpr52729.2023.02334.'
ieee: 'E. B. Iofinova, A. Krumes, and D.-A. Alistarh, “Bias in pruned vision models:
In-depth analysis and countermeasures,” in 2023 IEEE/CVF Conference on Computer
Vision and Pattern Recognition, Vancouver, BC, Canada, 2023, pp. 24364–24373.'
ista: 'Iofinova EB, Krumes A, Alistarh D-A. 2023. Bias in pruned vision models:
In-depth analysis and countermeasures. 2023 IEEE/CVF Conference on Computer Vision
and Pattern Recognition. CVPR: Conference on Computer Vision and Pattern Recognition,
24364–24373.'
mla: 'Iofinova, Eugenia B., et al. “Bias in Pruned Vision Models: In-Depth Analysis
and Countermeasures.” 2023 IEEE/CVF Conference on Computer Vision and Pattern
Recognition, IEEE, 2023, pp. 24364–73, doi:10.1109/cvpr52729.2023.02334.'
short: E.B. Iofinova, A. Krumes, D.-A. Alistarh, in:, 2023 IEEE/CVF Conference on
Computer Vision and Pattern Recognition, IEEE, 2023, pp. 24364–24373.
conference:
end_date: 2023-06-24
location: Vancouver, BC, Canada
name: 'CVPR: Conference on Computer Vision and Pattern Recognition'
start_date: 2023-06-17
corr_author: '1'
date_created: 2024-01-10T08:42:40Z
date_published: 2023-08-22T00:00:00Z
date_updated: 2026-05-19T11:20:27Z
day: '22'
department:
- _id: DaAl
- _id: ChLa
doi: 10.1109/cvpr52729.2023.02334
ec_funded: 1
external_id:
arxiv:
- '2304.12622'
isi:
- '001062531308068'
isi: 1
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2304.12622
month: '08'
oa: 1
oa_version: Preprint
page: 24364-24373
project:
- _id: 9B9290DE-BA93-11EA-9121-9846C619BF3A
grant_number: W1260-N35
name: Vienna Graduate School on Computational Optimization
- _id: 268A44D6-B435-11E9-9278-68D0E5697425
call_identifier: H2020
grant_number: '805223'
name: Elastic Coordination for Scalable Machine Learning
publication: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition
publication_identifier:
eisbn:
- '9798350301298'
eissn:
- 2575-7075
publication_status: published
publisher: IEEE
quality_controlled: '1'
related_material:
link:
- relation: software
url: https://github.com/IST-DASLab/pruned-vision-model-bias
record:
- id: '21854'
relation: dissertation_contains
status: public
status: public
title: 'Bias in pruned vision models: In-depth analysis and countermeasures'
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
year: '2023'
...
---
_id: '12299'
abstract:
- lang: eng
text: 'Transfer learning is a classic paradigm by which models pretrained on large
“upstream” datasets are adapted to yield good results on “downstream” specialized
datasets. Generally, more accurate models on the “upstream” dataset tend to provide
better transfer accuracy “downstream”. In this work, we perform an in-depth investigation
of this phenomenon in the context of convolutional neural networks (CNNs) trained
on the ImageNet dataset, which have been pruned-that is, compressed by sparsifiying
their connections. We consider transfer using unstructured pruned models obtained
by applying several state-of-the-art pruning methods, including magnitude-based,
second-order, regrowth, lottery-ticket, and regularization approaches, in the
context of twelve standard transfer tasks. In a nutshell, our study shows that
sparse models can match or even outperform the transfer performance of dense models,
even at high sparsities, and, while doing so, can lead to significant inference
and even training speedups. At the same time, we observe and analyze significant
differences in the behaviour of different pruning methods. The code is available
at: https://github.com/IST-DASLab/sparse-imagenet-transfer.'
acknowledgement: he authors would like to sincerely thank Christoph Lampert and Nir
Shavit for fruitful discussions during the development of this work, and Eldar Kurtic
for experimental support. EI was supported in part by the FWF DK VGSCO, grant agreement
number W1260-N35, while AP and DA acknowledge generous support by the ERC, via Starting
Grant 805223 ScaleML.
article_processing_charge: No
arxiv: 1
author:
- first_name: Eugenia B
full_name: Iofinova, Eugenia B
id: f9a17499-f6e0-11ea-865d-fdf9a3f77117
last_name: Iofinova
orcid: 0000-0002-7778-3221
- first_name: Elena-Alexandra
full_name: Peste, Elena-Alexandra
id: 32D78294-F248-11E8-B48F-1D18A9856A87
last_name: Peste
- first_name: Mark
full_name: Kurtz, Mark
last_name: Kurtz
- first_name: Dan-Adrian
full_name: Alistarh, Dan-Adrian
id: 4A899BFC-F248-11E8-B48F-1D18A9856A87
last_name: Alistarh
orcid: 0000-0003-3650-940X
citation:
ama: 'Iofinova EB, Krumes A, Kurtz M, Alistarh D-A. How well do sparse ImageNet
models transfer? In: 2022 IEEE/CVF Conference on Computer Vision and Pattern
Recognition. Institute of Electrical and Electronics Engineers; 2022:12256-12266.
doi:10.1109/cvpr52688.2022.01195'
apa: 'Iofinova, E. B., Krumes, A., Kurtz, M., & Alistarh, D.-A. (2022). How
well do sparse ImageNet models transfer? In 2022 IEEE/CVF Conference on Computer
Vision and Pattern Recognition (pp. 12256–12266). New Orleans, LA, United
States: Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/cvpr52688.2022.01195'
chicago: Iofinova, Eugenia B, Alexandra Krumes, Mark Kurtz, and Dan-Adrian Alistarh.
“How Well Do Sparse ImageNet Models Transfer?” In 2022 IEEE/CVF Conference
on Computer Vision and Pattern Recognition, 12256–66. Institute of Electrical
and Electronics Engineers, 2022. https://doi.org/10.1109/cvpr52688.2022.01195.
ieee: E. B. Iofinova, A. Krumes, M. Kurtz, and D.-A. Alistarh, “How well do sparse
ImageNet models transfer?,” in 2022 IEEE/CVF Conference on Computer Vision
and Pattern Recognition, New Orleans, LA, United States, 2022, pp. 12256–12266.
ista: 'Iofinova EB, Krumes A, Kurtz M, Alistarh D-A. 2022. How well do sparse ImageNet
models transfer? 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition.
CVPR: Computer Vision and Pattern Recognition, 12256–12266.'
mla: Iofinova, Eugenia B., et al. “How Well Do Sparse ImageNet Models Transfer?”
2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Institute
of Electrical and Electronics Engineers, 2022, pp. 12256–66, doi:10.1109/cvpr52688.2022.01195.
short: E.B. Iofinova, A. Krumes, M. Kurtz, D.-A. Alistarh, in:, 2022 IEEE/CVF Conference
on Computer Vision and Pattern Recognition, Institute of Electrical and Electronics
Engineers, 2022, pp. 12256–12266.
conference:
end_date: 2022-06-24
location: New Orleans, LA, United States
name: 'CVPR: Computer Vision and Pattern Recognition'
start_date: 2022-06-18
corr_author: '1'
date_created: 2023-01-16T10:06:00Z
date_published: 2022-09-27T00:00:00Z
date_updated: 2026-04-07T13:30:19Z
day: '27'
department:
- _id: DaAl
- _id: ChLa
doi: 10.1109/cvpr52688.2022.01195
ec_funded: 1
external_id:
arxiv:
- '2111.13445'
isi:
- '000870759105034'
isi: 1
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2111.13445
month: '09'
oa: 1
oa_version: Preprint
page: 12256-12266
project:
- _id: 9B9290DE-BA93-11EA-9121-9846C619BF3A
grant_number: W1260-N35
name: Vienna Graduate School on Computational Optimization
- _id: 268A44D6-B435-11E9-9278-68D0E5697425
call_identifier: H2020
grant_number: '805223'
name: Elastic Coordination for Scalable Machine Learning
publication: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition
publication_identifier:
eissn:
- 2575-7075
publication_status: published
publisher: Institute of Electrical and Electronics Engineers
quality_controlled: '1'
related_material:
record:
- id: '13074'
relation: dissertation_contains
status: public
scopus_import: '1'
status: public
title: How well do sparse ImageNet models transfer?
type: conference
user_id: 4359f0d1-fa6c-11eb-b949-802e58b17ae8
year: '2022'
...
---
_id: '12495'
abstract:
- lang: eng
text: "Fairness-aware learning aims at constructing classifiers that not only make
accurate predictions, but also do not discriminate against specific groups. It
is a fast-growing area of\r\nmachine learning with far-reaching societal impact.
However, existing fair learning methods\r\nare vulnerable to accidental or malicious
artifacts in the training data, which can cause\r\nthem to unknowingly produce
unfair classifiers. In this work we address the problem of\r\nfair learning from
unreliable training data in the robust multisource setting, where the\r\navailable
training data comes from multiple sources, a fraction of which might not be representative
of the true data distribution. We introduce FLEA, a filtering-based algorithm\r\nthat
identifies and suppresses those data sources that would have a negative impact
on\r\nfairness or accuracy if they were used for training. As such, FLEA is not
a replacement of\r\nprior fairness-aware learning methods but rather an augmentation
that makes any of them\r\nrobust against unreliable training data. We show the
effectiveness of our approach by a\r\ndiverse range of experiments on multiple
datasets. Additionally, we prove formally that\r\n–given enough data– FLEA protects
the learner against corruptions as long as the fraction of\r\naffected data sources
is less than half. Our source code and documentation are available at\r\nhttps://github.com/ISTAustria-CVML/FLEA."
acknowledged_ssus:
- _id: ScienComp
acknowledgement: 'The authors would like to thank Bernd Prach, Elias Frantar, Alexandra
Peste, Mahdi Nikdan, and Peter Súkeník for their helpful feedback. This research
was supported by the Scientific Service Units (SSU) of IST Austria through resources
provided by Scientific Computing (SciComp). This publication was made possible by
an ETH AI Center postdoctoral fellowship granted to Nikola Konstantinov. Eugenia
Iofinova was supported in part by the FWF DK VGSCO, grant agreement number W1260-N35. '
article_processing_charge: No
article_type: original
arxiv: 1
author:
- first_name: Eugenia B
full_name: Iofinova, Eugenia B
id: f9a17499-f6e0-11ea-865d-fdf9a3f77117
last_name: Iofinova
orcid: 0000-0002-7778-3221
- first_name: Nikola H
full_name: Konstantinov, Nikola H
id: 4B9D76E4-F248-11E8-B48F-1D18A9856A87
last_name: Konstantinov
orcid: 0009-0009-5204-7621
- first_name: Christoph
full_name: Lampert, Christoph
id: 40C20FD2-F248-11E8-B48F-1D18A9856A87
last_name: Lampert
orcid: 0000-0001-8622-7887
citation:
ama: 'Iofinova EB, Konstantinov NH, Lampert C. FLEA: Provably robust fair multisource
learning from unreliable training data. Transactions on Machine Learning Research.
2022.'
apa: 'Iofinova, E. B., Konstantinov, N. H., & Lampert, C. (2022). FLEA: Provably
robust fair multisource learning from unreliable training data. Transactions
on Machine Learning Research. ML Research Press.'
chicago: 'Iofinova, Eugenia B, Nikola H Konstantinov, and Christoph Lampert. “FLEA:
Provably Robust Fair Multisource Learning from Unreliable Training Data.” Transactions
on Machine Learning Research. ML Research Press, 2022.'
ieee: 'E. B. Iofinova, N. H. Konstantinov, and C. Lampert, “FLEA: Provably robust
fair multisource learning from unreliable training data,” Transactions on Machine
Learning Research. ML Research Press, 2022.'
ista: 'Iofinova EB, Konstantinov NH, Lampert C. 2022. FLEA: Provably robust fair
multisource learning from unreliable training data. Transactions on Machine Learning
Research.'
mla: 'Iofinova, Eugenia B., et al. “FLEA: Provably Robust Fair Multisource Learning
from Unreliable Training Data.” Transactions on Machine Learning Research,
ML Research Press, 2022.'
short: E.B. Iofinova, N.H. Konstantinov, C. Lampert, Transactions on Machine Learning
Research (2022).
corr_author: '1'
date_created: 2023-02-02T20:29:57Z
date_published: 2022-12-22T00:00:00Z
date_updated: 2025-12-30T11:04:31Z
day: '22'
ddc:
- '000'
department:
- _id: ChLa
external_id:
arxiv:
- '2106.11732'
file:
- access_level: open_access
checksum: 97c8a8470759cab597abb973ca137a3b
content_type: application/pdf
creator: dernst
date_created: 2023-02-23T10:30:04Z
date_updated: 2023-02-23T10:30:04Z
file_id: '12673'
file_name: 2022_TMLR_Iofinova.pdf
file_size: 1948063
relation: main_file
success: 1
file_date_updated: 2023-02-23T10:30:04Z
has_accepted_license: '1'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://openreview.net/forum?id=XsPopigZXV
month: '12'
oa: 1
oa_version: Published Version
project:
- _id: 9B9290DE-BA93-11EA-9121-9846C619BF3A
grant_number: W1260-N35
name: Vienna Graduate School on Computational Optimization
publication: Transactions on Machine Learning Research
publication_identifier:
issn:
- 2835-8856
publication_status: published
publisher: ML Research Press
quality_controlled: '1'
related_material:
link:
- description: source code
relation: software
url: https://github.com/ISTAustria-CVML/FLEA
status: public
title: 'FLEA: Provably robust fair multisource learning from unreliable training data'
tmp:
image: /images/cc_by.png
legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode
name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)
short: CC BY (4.0)
type: journal_article
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
year: '2022'
...
---
_id: '11458'
abstract:
- lang: eng
text: 'The increasing computational requirements of deep neural networks (DNNs)
have led to significant interest in obtaining DNN models that are sparse, yet
accurate. Recent work has investigated the even harder case of sparse training,
where the DNN weights are, for as much as possible, already sparse to reduce computational
costs during training. Existing sparse training methods are often empirical and
can have lower accuracy relative to the dense baseline. In this paper, we present
a general approach called Alternating Compressed/DeCompressed (AC/DC) training
of DNNs, demonstrate convergence for a variant of the algorithm, and show that
AC/DC outperforms existing sparse training methods in accuracy at similar computational
budgets; at high sparsity levels, AC/DC even outperforms existing methods that
rely on accurate pre-trained dense models. An important property of AC/DC is that
it allows co-training of dense and sparse models, yielding accurate sparse–dense
model pairs at the end of the training process. This is useful in practice, where
compressed variants may be desirable for deployment in resource-constrained settings
without re-doing the entire training flow, and also provides us with insights
into the accuracy gap between dense and compressed models. The code is available
at: https://github.com/IST-DASLab/ACDC.'
acknowledged_ssus:
- _id: ScienComp
acknowledgement: This project has received funding from the European Research Council
(ERC) under the European Union’s Horizon 2020 research and innovation programme
(grant agreement No 805223 ScaleML), and a CNRS PEPS grant. This research was supported
by the Scientific Service Units (SSU) of IST Austria through resources provided
by Scientific Computing (SciComp). We would also like to thank Christoph Lampert
for his feedback on an earlier version of this work, as well as for providing hardware
for the Transformer-XL experiments.
alternative_title:
- Advances in Neural Information Processing Systems
article_processing_charge: No
arxiv: 1
author:
- first_name: Elena-Alexandra
full_name: Peste, Elena-Alexandra
id: 32D78294-F248-11E8-B48F-1D18A9856A87
last_name: Peste
- first_name: Eugenia B
full_name: Iofinova, Eugenia B
id: f9a17499-f6e0-11ea-865d-fdf9a3f77117
last_name: Iofinova
orcid: 0000-0002-7778-3221
- first_name: Adrian
full_name: Vladu, Adrian
last_name: Vladu
- first_name: Dan-Adrian
full_name: Alistarh, Dan-Adrian
id: 4A899BFC-F248-11E8-B48F-1D18A9856A87
last_name: Alistarh
orcid: 0000-0003-3650-940X
citation:
ama: 'Krumes A, Iofinova EB, Vladu A, Alistarh D-A. AC/DC: Alternating Compressed/DeCompressed
training of deep neural networks. In: 35th Conference on Neural Information
Processing Systems. Vol 34. Neural Information Processing Systems Foundation;
2021:8557-8570.'
apa: 'Krumes, A., Iofinova, E. B., Vladu, A., & Alistarh, D.-A. (2021). AC/DC:
Alternating Compressed/DeCompressed training of deep neural networks. In 35th
Conference on Neural Information Processing Systems (Vol. 34, pp. 8557–8570).
Virtual, Online: Neural Information Processing Systems Foundation.'
chicago: 'Krumes, Alexandra, Eugenia B Iofinova, Adrian Vladu, and Dan-Adrian Alistarh.
“AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networks.”
In 35th Conference on Neural Information Processing Systems, 34:8557–70.
Neural Information Processing Systems Foundation, 2021.'
ieee: 'A. Krumes, E. B. Iofinova, A. Vladu, and D.-A. Alistarh, “AC/DC: Alternating
Compressed/DeCompressed training of deep neural networks,” in 35th Conference
on Neural Information Processing Systems, Virtual, Online, 2021, vol. 34,
pp. 8557–8570.'
ista: 'Krumes A, Iofinova EB, Vladu A, Alistarh D-A. 2021. AC/DC: Alternating Compressed/DeCompressed
training of deep neural networks. 35th Conference on Neural Information Processing
Systems. NeurIPS: Neural Information Processing Systems, Advances in Neural Information
Processing Systems, vol. 34, 8557–8570.'
mla: 'Krumes, Alexandra, et al. “AC/DC: Alternating Compressed/DeCompressed Training
of Deep Neural Networks.” 35th Conference on Neural Information Processing
Systems, vol. 34, Neural Information Processing Systems Foundation, 2021,
pp. 8557–70.'
short: A. Krumes, E.B. Iofinova, A. Vladu, D.-A. Alistarh, in:, 35th Conference
on Neural Information Processing Systems, Neural Information Processing Systems
Foundation, 2021, pp. 8557–8570.
conference:
end_date: 2021-12-14
location: Virtual, Online
name: 'NeurIPS: Neural Information Processing Systems'
start_date: 2021-12-06
corr_author: '1'
date_created: 2022-06-20T12:11:53Z
date_published: 2021-12-06T00:00:00Z
date_updated: 2026-06-18T17:18:20Z
day: '06'
ddc:
- '000'
department:
- _id: GradSch
- _id: DaAl
ec_funded: 1
external_id:
arxiv:
- '2106.12379'
intvolume: ' 34'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://proceedings.neurips.cc/paper/2021/file/48000647b315f6f00f913caa757a70b3-Paper.pdf
month: '12'
oa: 1
oa_version: Published Version
page: 8557-8570
project:
- _id: 268A44D6-B435-11E9-9278-68D0E5697425
call_identifier: H2020
grant_number: '805223'
name: Elastic Coordination for Scalable Machine Learning
publication: 35th Conference on Neural Information Processing Systems
publication_identifier:
isbn:
- '9781713845393'
issn:
- 1049-5258
publication_status: published
publisher: Neural Information Processing Systems Foundation
quality_controlled: '1'
related_material:
record:
- id: '13074'
relation: dissertation_contains
status: public
scopus_import: '1'
status: public
title: 'AC/DC: Alternating Compressed/DeCompressed training of deep neural networks'
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 34
year: '2021'
...