---
APC_amount: 1470 EUR
DOAJ_listed: '1'
OA_place: publisher
OA_type: gold
_id: '15165'
abstract:
- lang: eng
text: Current knowledge suggests a drought Indian monsoon (perhaps a severe one)
when the El Nino Southern Oscillation and Pacific Decadal Oscillation each exhibit
positive phases (a joint positive phase). For the monsoons, which are exceptions
in this regard, we found northeast India often gets excess pre-monsoon rainfall.
Further investigation reveals that this excess pre-monsoon rainfall is produced
by the interaction of the large-scale circulation associated with the joint phase
with the mountains in northeast India. We posit that a warmer troposphere, a consequence
of excess rainfall over northeast India, drives a stronger monsoon circulation
and enhances monsoon rainfall over central India. Hence, we argue that pre-monsoon
rainfall over northeast India can be used for seasonal monsoon rainfall prediction
over central India. Most importantly, its predictive value is at its peak when
the Pacific Ocean exhibits a joint positive phase and the threat of extreme drought
monsoon looms over India.
acknowledgement: The author gratefully acknowledges ISTA for supporting this research
through funding from the European Research Council (ERC) under the European Union’s
Horizon 2020 research and innovation programme (Project CLUSTER, grant agreement
No. 805041).
article_number: e2023GL106569
article_processing_charge: Yes
article_type: original
author:
- first_name: Bidyut B
full_name: Goswami, Bidyut B
id: 3a4ac09c-6d61-11ec-bf66-884cde66b64b
last_name: Goswami
orcid: 0000-0001-8602-3083
citation:
ama: GOSWAMI BB. A pre-monsoon signal of false alarms of Indian monsoon droughts.
Geophysical Research Letters. 2024;51(5). doi:10.1029/2023GL106569
apa: GOSWAMI, B. B. (2024). A pre-monsoon signal of false alarms of Indian monsoon
droughts. Geophysical Research Letters. Wiley. https://doi.org/10.1029/2023GL106569
chicago: GOSWAMI, BIDYUT B. “A Pre-Monsoon Signal of False Alarms of Indian Monsoon
Droughts.” Geophysical Research Letters. Wiley, 2024. https://doi.org/10.1029/2023GL106569.
ieee: B. B. GOSWAMI, “A pre-monsoon signal of false alarms of Indian monsoon droughts,”
Geophysical Research Letters, vol. 51, no. 5. Wiley, 2024.
ista: GOSWAMI BB. 2024. A pre-monsoon signal of false alarms of Indian monsoon droughts.
Geophysical Research Letters. 51(5), e2023GL106569.
mla: GOSWAMI, BIDYUT B. “A Pre-Monsoon Signal of False Alarms of Indian Monsoon
Droughts.” Geophysical Research Letters, vol. 51, no. 5, e2023GL106569,
Wiley, 2024, doi:10.1029/2023GL106569.
short: B.B. GOSWAMI, Geophysical Research Letters 51 (2024).
corr_author: '1'
das_tickbox: '1'
dataavailabilitystatement: The observed rainfall data analyzed in this study are from
the IMD (Pai et al., 2015), and Tropical Rainfall Measurement Mission (TRMM) Multi-Satellite
Precipitation Analysis (TMPA) 3B42 Version 7 product (Huffman et al., 2007), reanalysis
data from 5th generation ECMWF reanalysis product (ERA5) (Hersbach et al., 2023),
SST data from the HadISST1 data set provided by the Met Office Hadley Center (Rayner
et al., 2003), available at https://www.metoffice.gov.uk/hadobs/hadisst/. The PDO
index, computed following (Mantua et al., 1997; Zhang et al., 1997) using the UKMO
Historical SST data set for 1900–81 (Parker et al., 1995); Reynold's Optimally Interpolated
(OI) SST (V1) for January 1982–Dec 2001) (Reynolds et al., 2007) and OI SST Version
2 (V2) beginning January 2002–present, is obtained from the PDO web-page maintained
by Dr. Nathan Mantua, NOAA Fisheries, available at http://research.jisao.washington.edu/pdo/PDO.latest.txt.
The ENSO index, monthly NINO3.4 values, computed from HadISST1 data (Rayner et al.,
2003), is obtained from the Global Climate Observing System (GCOS) Working Group
on Surface Pressure (WG-SP), web-page hosted by NOAA Physical Sciences Laboratory
(PSL), available at https://psl.noaa.gov/gcos_wgsp/Timeseries/Data/nino34.long.anom.data.
The linear regression model, that we constructed, is based on the LinearRegression
function from sklearn Python package; the python script for this regression analysis
is available at (Goswami, 2023).
date_created: 2024-03-24T23:00:58Z
date_published: 2024-03-16T00:00:00Z
date_updated: 2026-10-01T09:17:49Z
day: '16'
ddc:
- '550'
department:
- _id: CaMu
doi: 10.1029/2023GL106569
ec_funded: 1
external_id:
isi:
- '001181635700001'
file:
- access_level: open_access
checksum: 243bd966aca968ec7d9e474af8639f8d
content_type: application/pdf
creator: dernst
date_created: 2024-03-25T08:36:00Z
date_updated: 2024-03-25T08:36:00Z
file_id: '15178'
file_name: 2024_GeophysResLetters_Goswami.pdf
file_size: 2887134
relation: main_file
success: 1
file_date_updated: 2024-03-25T08:36:00Z
fulldoi: https://doi.org/10.1029/2023GL106569
has_accepted_license: '1'
intvolume: ' 51'
isi: 1
issue: '5'
language:
- iso: eng
license: https://creativecommons.org/licenses/by-nc-nd/4.0/
month: '03'
oa: 1
oa_version: Published Version
project:
- _id: 629205d8-2b32-11ec-9570-e1356ff73576
call_identifier: H2020
grant_number: '805041'
name: Organization of CLoUdS, and implications of Tropical cyclones and for the
Energetics of the tropics, in current and waRming climate
publication: Geophysical Research Letters
publication_identifier:
eissn:
- 1944-8007
issn:
- 0094-8276
publication_status: published
publisher: Wiley
quality_controlled: '1'
researchdata_availability: not applicable
scopus_import: '1'
status: public
supplementarymaterial: yes
title: A pre-monsoon signal of false alarms of Indian monsoon droughts
tmp:
image: /images/cc_by_nc_nd.png
legal_code_url: https://creativecommons.org/licenses/by-nc-nd/4.0/legalcode
name: Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International
(CC BY-NC-ND 4.0)
short: CC BY-NC-ND (4.0)
type: journal_article
user_id: 317138e5-6ab7-11ef-aa6d-ffef3953e345
volume: 51
year: '2024'
...
---
DOAJ_listed: '1'
OA_place: publisher
OA_type: gold
_id: '18708'
abstract:
- lang: eng
text: This study investigates the response of Indian summer monsoon (ISM) precipitation
to CO2 removal, with a specific focus on regional and subseasonal variations.
Following CO2 removal, monsoon circulation weakens throughout the summer owing
to the reduced large-scale meridional temperature gradient around India. Weakened
monsoon circulation decreases the local-scale thermodynamic stability within India,
following monsoon-onset periods. While the frequency of synoptic-scale ISM low-pressure
systems (LPSs) decreases overall, the lower thermodynamic stability causes the
LPSs to form and resultantly shift west and south from their typical paths, last
longer and move more quickly zonally during August and September. Changes in these
rain-producing processes induce distinct regional (Western Ghats, south-central-east
India, and Tamil Nadu) and subseasonal precipitation responses. Also, extreme
precipitation exhibits similar patterns, but is more strongly affected by changes
in LPS. Our results suggest that reliable future projections of regional hydroclimate
change require a more accurate understanding of multi-scale precipitation processes.
acknowledgement: This study was supported by the National Research Foundation of Korea
(NRF) grant funded by the Korean government (MSIT) (NRF-2018R1A5A1024958, NRF-2021R1C1C2094185,
RS-2024-00336160). Model simulation and data transfer were supported by the National
Supercomputing Center with supercomputing resources including technical support
(KSC-2021-CHA-0030), the National Center for Meteorological Supercomputer of the
Korea Meteorological Administration (KMA), and by the Korea Research Environment
Open NETwork (KREONET), respectively. DK was supported by New Faculty Startup Fund
from Seoul National University. We acknowledge the World Climate Research Programme,
which, through its Working Group on Coupled Modelling, coordinated and promoted
CMIP6. We thank the climate modeling groups for producing and making available their
model output, the Earth System Grid Federation (ESGF) for archiving the data and
providing access (https://esgf-node.llnl.gov/projects/cmip6/), and the multiple
funding agencies who support CMIP6 and ESGF.
article_number: '305'
article_processing_charge: Yes
article_type: original
author:
- first_name: Seungmok
full_name: Paik, Seungmok
last_name: Paik
- first_name: Daehyun
full_name: Kim, Daehyun
last_name: Kim
- first_name: Soon Il
full_name: An, Soon Il
last_name: An
- first_name: Hyoeun
full_name: Oh, Hyoeun
last_name: Oh
- first_name: Jongsoo
full_name: Shin, Jongsoo
last_name: Shin
- first_name: Bidyut B
full_name: Goswami, Bidyut B
id: 3a4ac09c-6d61-11ec-bf66-884cde66b64b
last_name: Goswami
orcid: 0000-0001-8602-3083
- first_name: Seung Ki
full_name: Min, Seung Ki
last_name: Min
- first_name: Sanjit Kumar
full_name: Mondal, Sanjit Kumar
last_name: Mondal
citation:
ama: Paik S, Kim D, An SI, et al. Exploring causes of distinct regional and subseasonal
Indian summer monsoon precipitation responses to CO2 removal. npj Climate and
Atmospheric Science. 2024;7. doi:10.1038/s41612-024-00858-0
apa: Paik, S., Kim, D., An, S. I., Oh, H., Shin, J., GOSWAMI, B. B., … Mondal, S.
K. (2024). Exploring causes of distinct regional and subseasonal Indian summer
monsoon precipitation responses to CO2 removal. Npj Climate and Atmospheric
Science. Springer Nature. https://doi.org/10.1038/s41612-024-00858-0
chicago: Paik, Seungmok, Daehyun Kim, Soon Il An, Hyoeun Oh, Jongsoo Shin, BIDYUT
B GOSWAMI, Seung Ki Min, and Sanjit Kumar Mondal. “Exploring Causes of Distinct
Regional and Subseasonal Indian Summer Monsoon Precipitation Responses to CO2
Removal.” Npj Climate and Atmospheric Science. Springer Nature, 2024. https://doi.org/10.1038/s41612-024-00858-0.
ieee: S. Paik et al., “Exploring causes of distinct regional and subseasonal
Indian summer monsoon precipitation responses to CO2 removal,” npj Climate
and Atmospheric Science, vol. 7. Springer Nature, 2024.
ista: Paik S, Kim D, An SI, Oh H, Shin J, GOSWAMI BB, Min SK, Mondal SK. 2024. Exploring
causes of distinct regional and subseasonal Indian summer monsoon precipitation
responses to CO2 removal. npj Climate and Atmospheric Science. 7, 305.
mla: Paik, Seungmok, et al. “Exploring Causes of Distinct Regional and Subseasonal
Indian Summer Monsoon Precipitation Responses to CO2 Removal.” Npj Climate
and Atmospheric Science, vol. 7, 305, Springer Nature, 2024, doi:10.1038/s41612-024-00858-0.
short: S. Paik, D. Kim, S.I. An, H. Oh, J. Shin, B.B. GOSWAMI, S.K. Min, S.K. Mondal,
Npj Climate and Atmospheric Science 7 (2024).
das_tickbox: '1'
dataavailabilitystatement: CESM1 simulation datasets supporting the results will be
provided upon a reasonable request. Global Precipitation Climatology Project version
2.3 can be accessed at https://www.ncei.noaa.gov/data/global-precipitation-climatology-project-gpcp-monthly/access/.
ERA5 datasets can be accessed at https://doi.org/10.24381/cds.bd0915c6, https://doi.org/10.24381/cds.adbb2d47
and https://doi.org/10.24381/cds.6860a573. Global Precipitation Climatology Centre
version 2022 can be accessed at https://opendata.dwd.de/climate_environment/GPCC/html/fulldata-monthly_v2022_doi_download.html.
All of the CMIP6 model simulation datasets used in this study can be accessed at
https://esgf-node.llnl.gov/search/cmip6/.
date_created: 2024-12-29T23:01:57Z
date_published: 2024-12-19T00:00:00Z
date_updated: 2026-10-01T09:26:16Z
day: '19'
ddc:
- '550'
department:
- _id: CaMu
doi: 10.1038/s41612-024-00858-0
external_id:
isi:
- '001381218300007'
file:
- access_level: open_access
checksum: 6b3148315a444835113c32b399010370
content_type: application/pdf
creator: dernst
date_created: 2025-01-02T08:49:13Z
date_updated: 2025-01-02T08:49:13Z
file_id: '18717'
file_name: 2024_npjclimate_Paik.pdf
file_size: 1927871
relation: main_file
success: 1
file_date_updated: 2025-01-02T08:49:13Z
fulldoi: https://doi.org/10.1038/s41612-024-00858-0
has_accepted_license: '1'
intvolume: ' 7'
isi: 1
language:
- iso: eng
month: '12'
oa: 1
oa_version: Published Version
publication: npj Climate and Atmospheric Science
publication_identifier:
eissn:
- 2397-3722
publication_status: published
publisher: Springer Nature
quality_controlled: '1'
researchdata_availability: upon request
scopus_import: '1'
status: public
supplementarymaterial: yes
title: Exploring causes of distinct regional and subseasonal Indian summer monsoon
precipitation responses to CO2 removal
tmp:
image: /images/cc_by_nc_nd.png
legal_code_url: https://creativecommons.org/licenses/by-nc-nd/4.0/legalcode
name: Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International
(CC BY-NC-ND 4.0)
short: CC BY-NC-ND (4.0)
type: journal_article
user_id: 317138e5-6ab7-11ef-aa6d-ffef3953e345
volume: 7
year: '2024'
...
---
OA_place: repository
OA_type: green
_id: '19307'
abstract:
- lang: eng
text: "This repository contains the data, scripts, SAM codes and files required
to reproduce the results of the manuscript \"The Unreasonable Efficiency of Total
Rain Evaporation Removal in Triggering Convective Self-Aggregation\" submitted
to the Geophysical Research Letters (GRL).\r\n\r\nBrief description of project:
This project aims to examine the impact of rain evaporation removal or reduction
in the planetary boundary layer (PBL) on convective self aggregation (CSA). Non-rotating
radiative-convective equilibrium (RCE) simulations were conducted with the System
for Atmospheric Modeling (SAM) cloud resolving model. Rain evaporation in the
lowest 1 km was progressively reduced and the effect on CSA was investigated.
The physical processes underlying this type of aggregation (referred to in the
manuscript as no-evaporation CSA, or NE-CSA) were analyzed and described. \r\nThe
default SAM code base (version 6.10.8) can be downloaded from here: http://rossby.msrc.sunysb.edu/~marat/SAM.html"
article_processing_charge: No
author:
- first_name: Yi-Ling
full_name: Hwong, Yi-Ling
id: 1217aa61-4dd1-11ec-9ac3-f2ba3f17ee22
last_name: Hwong
orcid: 0000-0001-9281-3479
- first_name: Caroline J
full_name: Muller, Caroline J
id: f978ccb0-3f7f-11eb-b193-b0e2bd13182b
last_name: Muller
orcid: 0000-0001-5836-5350
citation:
ama: Hwong Y-L, Muller CJ. Data - The unreasonable efficiency of total rain evaporation
removal in triggering convective self-aggregation. 2024. doi:10.5281/ZENODO.10687169
apa: Hwong, Y.-L., & Muller, C. J. (2024). Data - The unreasonable efficiency
of total rain evaporation removal in triggering convective self-aggregation. Zenodo.
https://doi.org/10.5281/ZENODO.10687169
chicago: Hwong, Yi-Ling, and Caroline J Muller. “Data - The Unreasonable Efficiency
of Total Rain Evaporation Removal in Triggering Convective Self-Aggregation.”
Zenodo, 2024. https://doi.org/10.5281/ZENODO.10687169.
ieee: Y.-L. Hwong and C. J. Muller, “Data - The unreasonable efficiency of total
rain evaporation removal in triggering convective self-aggregation.” Zenodo, 2024.
ista: Hwong Y-L, Muller CJ. 2024. Data - The unreasonable efficiency of total rain
evaporation removal in triggering convective self-aggregation, Zenodo, 10.5281/ZENODO.10687169.
mla: Hwong, Yi-Ling, and Caroline J. Muller. Data - The Unreasonable Efficiency
of Total Rain Evaporation Removal in Triggering Convective Self-Aggregation.
Zenodo, 2024, doi:10.5281/ZENODO.10687169.
short: Y.-L. Hwong, C.J. Muller, (2024).
corr_author: '1'
date_created: 2025-03-07T08:39:40Z
date_published: 2024-02-21T00:00:00Z
date_updated: 2026-10-01T09:22:28Z
day: '21'
ddc:
- '550'
department:
- _id: CaMu
doi: 10.5281/ZENODO.10687169
fulldoi: https://doi.org/10.5281/ZENODO.10687169
has_accepted_license: '1'
license: https://creativecommons.org/licenses/by/4.0/
main_file_link:
- open_access: '1'
url: https://doi.org/10.5281/zenodo.8369509
month: '02'
oa: 1
oa_version: Published Version
publisher: Zenodo
related_material:
record:
- id: '15186'
relation: used_in_publication
status: public
status: public
title: Data - The unreasonable efficiency of total rain evaporation removal in triggering
convective self-aggregation
tmp:
image: /images/cc_by.png
legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode
name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)
short: CC BY (4.0)
type: research_data_reference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
year: '2024'
...
---
_id: '17203'
abstract:
- lang: eng
text: "The behavior of a rigid body primarily depends on its mass moments, which
consist of the mass, center of mass, and moments of inertia. It is possible to
manipulate these quantities without altering the geometric appearance of an object
by introducing cavities in its interior. Algorithms that find cavities of suitable
shapes and sizes have enabled the computational design of spinning tops, yo-yos,
wheels, buoys, and statically balanced objects. Previous work is based, for example,
on topology optimization on voxel grids, which introduces a large number of optimization
variables and box constraints, or offset surface computation, which cannot guarantee
that solutions to a feasible problem will always be found.\r\n\r\nIn this work,
we provide a mathematical analysis of constrained topology optimization problems
that depend only on mass moments. This class of problems covers, among others,
all applications mentioned above. Our main result is to show that no matter the
outer shape of the rigid body to be optimized or the optimization objective and
constraints considered, the optimal solution always features a quadric-shaped
interface between material and cavities. This proves that optimal interfaces are
always ellipsoids, hyperboloids, paraboloids, or one of a few degenerate cases,
such as planes.\r\n\r\nThis insight lets us replace a difficult topology optimization
problem with a provably equivalent non-linear equation system in a small number
(<10) of variables, which represent the coefficients of the quadric. This system
can be solved in a few seconds for most examples, provides insights into the geometric
structure of many specific applications, and lets us describe their solution properties.
Finally, our method integrates seamlessly into modern fabrication workflows because
our solutions are analytical surfaces that are native to the CAD domain."
acknowledgement: We thank Gianmarco Cherchi for his help in tailoring the Mesh Booleans
code for this project, Stefan Jeschke for his help with the photographs, Malina
Strugaru and Aleksei Kalinov for their help with the samples, and the anonymous
reviewers as well as the members of the ISTA Visual Computing Group for their feedback.
This project was funded in part by the European Research Council (ERC Consolidator
Grant 101045083 CoDiNA).
article_number: '78'
article_processing_charge: Yes (via OA deal)
article_type: original
author:
- first_name: Christian
full_name: Hafner, Christian
id: 400429CC-F248-11E8-B48F-1D18A9856A87
last_name: Hafner
- first_name: Mickaël
full_name: Ly, Mickaël
id: 6340d7f0-b48d-11eb-b10d-b7487e71d9f1
last_name: Ly
- first_name: Christopher J
full_name: Wojtan, Christopher J
id: 3C61F1D2-F248-11E8-B48F-1D18A9856A87
last_name: Wojtan
orcid: 0000-0001-6646-5546
citation:
ama: 'Hafner C, Ly M, Wojtan C. Spin-it faster: Quadrics solve all topology optimization
problems that depend only on mass moments. Transactions on Graphics. 2024;43(4).
doi:10.1145/3658194'
apa: 'Hafner, C., Ly, M., & Wojtan, C. (2024). Spin-it faster: Quadrics solve
all topology optimization problems that depend only on mass moments. Transactions
on Graphics. Denver, Colorado: Association for Computing Machinery. https://doi.org/10.1145/3658194'
chicago: 'Hafner, Christian, Mickaël Ly, and Chris Wojtan. “Spin-It Faster: Quadrics
Solve All Topology Optimization Problems That Depend Only on Mass Moments.” Transactions
on Graphics. Association for Computing Machinery, 2024. https://doi.org/10.1145/3658194.'
ieee: 'C. Hafner, M. Ly, and C. Wojtan, “Spin-it faster: Quadrics solve all topology
optimization problems that depend only on mass moments,” Transactions on Graphics,
vol. 43, no. 4. Association for Computing Machinery, 2024.'
ista: 'Hafner C, Ly M, Wojtan C. 2024. Spin-it faster: Quadrics solve all topology
optimization problems that depend only on mass moments. Transactions on Graphics.
43(4), 78.'
mla: 'Hafner, Christian, et al. “Spin-It Faster: Quadrics Solve All Topology Optimization
Problems That Depend Only on Mass Moments.” Transactions on Graphics, vol.
43, no. 4, 78, Association for Computing Machinery, 2024, doi:10.1145/3658194.'
short: C. Hafner, M. Ly, C. Wojtan, Transactions on Graphics 43 (2024).
conference:
end_date: 2024-08-01
location: Denver, Colorado
start_date: 2024-07-28
corr_author: '1'
das_tickbox: '0'
date_created: 2024-07-05T12:08:57Z
date_published: 2024-07-01T00:00:00Z
date_updated: 2026-10-01T09:30:01Z
day: '01'
ddc:
- '516'
department:
- _id: ChWo
doi: 10.1145/3658194
external_id:
isi:
- '001289270900045'
file:
- access_level: open_access
checksum: 0dc9f5a6422b8a49a79026900f349ee5
content_type: application/pdf
creator: chafner
date_created: 2024-07-05T12:05:17Z
date_updated: 2024-07-05T12:05:17Z
file_id: '17204'
file_name: sif-final.pdf
file_size: 7225150
relation: main_file
success: 1
- access_level: open_access
checksum: cde433c6a40688d5f1187fb5721f6f94
content_type: application/pdf
creator: chafner
date_created: 2024-07-05T12:06:03Z
date_updated: 2024-07-05T12:06:03Z
file_id: '17205'
file_name: sif-supp-final.pdf
file_size: 397262
relation: supplementary_material
- access_level: open_access
checksum: c0457a09c2ab9a1c2935c995dcc84907
content_type: video/mp4
creator: chafner
date_created: 2024-07-17T09:29:13Z
date_updated: 2024-07-17T09:29:13Z
file_id: '17276'
file_name: sif-video-final.mp4
file_size: 170001305
relation: supplementary_material
title: Submission Video
file_date_updated: 2024-07-17T09:29:13Z
fulldoi: https://doi.org/10.1145/3658194
has_accepted_license: '1'
intvolume: ' 43'
isi: 1
issue: '4'
keyword:
- Topology Optimization
- Mass Moments
- Computational Geometry
language:
- iso: eng
month: '07'
oa: 1
oa_version: Published Version
project:
- _id: 34bc2376-11ca-11ed-8bc3-9a3b3961a088
grant_number: '101045083'
name: Computational Discovery of Numerical Algorithms for Animation and Simulation
of Natural Phenomena
publication: Transactions on Graphics
publication_identifier:
eissn:
- 1557-7368
issn:
- 0730-0301
publication_status: published
publisher: Association for Computing Machinery
quality_controlled: '1'
researchdata_availability: no
scopus_import: '1'
status: public
supplementarymaterial: yes
title: 'Spin-it faster: Quadrics solve all topology optimization problems that depend
only on mass moments'
type: journal_article
user_id: 317138e5-6ab7-11ef-aa6d-ffef3953e345
volume: 43
year: '2024'
...
---
OA_place: publisher
OA_type: hybrid
_id: '15313'
abstract:
- lang: eng
text: Our goal is to investigate fundamental properties of the system of internally
cooled convection. The system consists of an upward thermal flux at the lower
boundary, a mean temperature lapse-rate and a constant cooling term in the bulk
with the bulk cooling in thermal equilibrium with the input heat flux. This simple
model represents idealised dry convection in the atmospheric boundary layer, where
the cooling mimics the radiative cooling to space notably through longwave radiation.
We perform linear stability analysis of the model for different values of the
mean stratification to derive the critical forcing above which the fluid is convectively
unstable to small perturbations. The dynamic behavior of the fluid system is described
and the scaling of various important measured quantities such as the total vertical
convective heat flux and the upward mass flux is measured. We introduce a lapse-rate
dependent dimensionless Rayleigh-number Ray that determines the behavior of the
system, finding that the convective heat-flux and mass-flux scale approximately
as Ray0.5 and Ray0.7 respectively. The area-fraction of the domain that is occupied
by upward and downward moving fluid and the skewness of the vertical velocity
are studied to understand the asymmetry inherent in the system. We conclude with
a short discussion on the relevance to atmospheric convection and the scope for
further investigations of atmospheric convection using similar simplified approaches.
acknowledged_ssus:
- _id: ScienComp
acknowledgement: This project has received funding from the European Union’s Horizon
2020 research and innovation programme under the Marie Sklodowska–Curie grant agreement
No. 101034413. CM gratefully acknowledges funding from the European Research Council
(ERC) under the European Union’s Horizon 2020 research and innovation program (Project
CLUSTER, Grant Agreement No. 805041). This research was supported by the Scientific
Service Units (SSU) of IST Austria through resources provided by Scientific Computing
(SciComp).
article_number: '108011'
article_processing_charge: Yes (via OA deal)
article_type: original
arxiv: 1
author:
- first_name: Lokahith N
full_name: Agasthya, Lokahith N
id: cd100965-0804-11ed-9c55-f4878ff4e877
last_name: Agasthya
- first_name: Caroline J
full_name: Muller, Caroline J
id: f978ccb0-3f7f-11eb-b193-b0e2bd13182b
last_name: Muller
orcid: 0000-0001-5836-5350
citation:
ama: Agasthya LN, Muller CJ. Dynamics and scaling of internally cooled convection.
Communications in Nonlinear Science and Numerical Simulation. 2024;134.
doi:10.1016/j.cnsns.2024.108011
apa: Agasthya, L. N., & Muller, C. J. (2024). Dynamics and scaling of internally
cooled convection. Communications in Nonlinear Science and Numerical Simulation.
Elsevier. https://doi.org/10.1016/j.cnsns.2024.108011
chicago: Agasthya, Lokahith N, and Caroline J Muller. “Dynamics and Scaling of Internally
Cooled Convection.” Communications in Nonlinear Science and Numerical Simulation.
Elsevier, 2024. https://doi.org/10.1016/j.cnsns.2024.108011.
ieee: L. N. Agasthya and C. J. Muller, “Dynamics and scaling of internally cooled
convection,” Communications in Nonlinear Science and Numerical Simulation,
vol. 134. Elsevier, 2024.
ista: Agasthya LN, Muller CJ. 2024. Dynamics and scaling of internally cooled convection.
Communications in Nonlinear Science and Numerical Simulation. 134, 108011.
mla: Agasthya, Lokahith N., and Caroline J. Muller. “Dynamics and Scaling of Internally
Cooled Convection.” Communications in Nonlinear Science and Numerical Simulation,
vol. 134, 108011, Elsevier, 2024, doi:10.1016/j.cnsns.2024.108011.
short: L.N. Agasthya, C.J. Muller, Communications in Nonlinear Science and Numerical
Simulation 134 (2024).
corr_author: '1'
das_tickbox: '1'
dataavailabilitystatement: Data will be made available on request
date_created: 2024-04-14T22:01:01Z
date_published: 2024-07-01T00:00:00Z
date_updated: 2026-10-01T09:27:13Z
day: '01'
ddc:
- '550'
department:
- _id: CaMu
doi: 10.1016/j.cnsns.2024.108011
ec_funded: 1
external_id:
arxiv:
- '2311.04114'
isi:
- '001238294600001'
file:
- access_level: open_access
checksum: 9b7c2b8281d0b7bc7f08e0468168324c
content_type: application/pdf
creator: dernst
date_created: 2025-01-09T09:05:31Z
date_updated: 2025-01-09T09:05:31Z
file_id: '18795'
file_name: 2024_CommNonlinear_Agasthya.pdf
file_size: 1257925
relation: main_file
success: 1
file_date_updated: 2025-01-09T09:05:31Z
fulldoi: https://doi.org/10.1016/j.cnsns.2024.108011
has_accepted_license: '1'
intvolume: ' 134'
isi: 1
language:
- iso: eng
month: '07'
oa: 1
oa_version: Published Version
project:
- _id: fc2ed2f7-9c52-11eb-aca3-c01059dda49c
call_identifier: H2020
grant_number: '101034413'
name: 'IST-BRIDGE: International postdoctoral program'
- _id: 629205d8-2b32-11ec-9570-e1356ff73576
call_identifier: H2020
grant_number: '805041'
name: Organization of CLoUdS, and implications of Tropical cyclones and for the
Energetics of the tropics, in current and waRming climate
publication: Communications in Nonlinear Science and Numerical Simulation
publication_identifier:
issn:
- 1007-5704
publication_status: published
publisher: Elsevier
quality_controlled: '1'
researchdata_availability: upon request
scopus_import: '1'
status: public
supplementarymaterial: no
title: Dynamics and scaling of internally cooled convection
tmp:
image: /images/cc_by.png
legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode
name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)
short: CC BY (4.0)
type: journal_article
user_id: 317138e5-6ab7-11ef-aa6d-ffef3953e345
volume: 134
year: '2024'
...
---
_id: '17411'
abstract:
- lang: eng
text: "We present PeFLL, a new personalized federated learning algorithm that improves\r\nover
the state-of-the-art in three aspects: 1) it produces more accurate models,\r\nespecially
in the low-data regime, and not only for clients present during its\r\ntraining
phase, but also for any that may emerge in the future; 2) it reduces the\r\namount
of on-client computation and client-server communication by providing\r\nfuture
clients with ready-to-use personalized models that require no additional\r\nfinetuning
or optimization; 3) it comes with theoretical guarantees that establish\r\ngeneralization
from the observed clients to future ones.\r\nAt the core of PeFLL lies a learning-to-learn
approach that jointly trains an\r\nembedding network and a hypernetwork. The embedding
network is used to\r\nrepresent clients in a latent descriptor space in a way
that reflects their similarity\r\nto each other. The hypernetwork takes as input
such descriptors and outputs the\r\nparameters of fully personalized client models.
In combination, both networks\r\nconstitute a learning algorithm that achieves
state-of-the-art performance in several\r\npersonalized federated learning benchmarks"
acknowledged_ssus:
- _id: ScienComp
acknowledgement: "This research was supported by the Scientific Service Units (SSU)
of ISTA through resources provided by Scientific Computing (SciComp).\r\n"
article_processing_charge: No
arxiv: 1
author:
- first_name: Jonathan A
full_name: Scott, Jonathan A
id: e499926b-f6e0-11ea-865d-9c63db0031e8
last_name: Scott
- first_name: Hossein
full_name: Zakerinia, Hossein
id: 653bd8b6-f394-11eb-9cf6-c0bbf6cd78d4
last_name: Zakerinia
orcid: 0009-0007-3977-6462
- first_name: Christoph
full_name: Lampert, Christoph
id: 40C20FD2-F248-11E8-B48F-1D18A9856A87
last_name: Lampert
orcid: 0000-0001-8622-7887
citation:
ama: 'Scott JA, Zakerinia H, Lampert C. PEFLL: Personalized federated learning by
learning to learn. In: 12th International Conference on Learning Representations.
OpenReview; 2024.'
apa: 'Scott, J. A., Zakerinia, H., & Lampert, C. (2024). PEFLL: Personalized
federated learning by learning to learn. In 12th International Conference on
Learning Representations. Vienna, Austria: OpenReview.'
chicago: 'Scott, Jonathan A, Hossein Zakerinia, and Christoph Lampert. “PEFLL: Personalized
Federated Learning by Learning to Learn.” In 12th International Conference
on Learning Representations. OpenReview, 2024.'
ieee: 'J. A. Scott, H. Zakerinia, and C. Lampert, “PEFLL: Personalized federated
learning by learning to learn,” in 12th International Conference on Learning
Representations, Vienna, Austria, 2024.'
ista: 'Scott JA, Zakerinia H, Lampert C. 2024. PEFLL: Personalized federated learning
by learning to learn. 12th International Conference on Learning Representations.
ICLR: International Conference on Learning Representations.'
mla: 'Scott, Jonathan A., et al. “PEFLL: Personalized Federated Learning by Learning
to Learn.” 12th International Conference on Learning Representations, OpenReview,
2024.'
short: J.A. Scott, H. Zakerinia, C. Lampert, in:, 12th International Conference
on Learning Representations, OpenReview, 2024.
conference:
end_date: 2024-03-07
location: Vienna, Austria
name: 'ICLR: International Conference on Learning Representations'
start_date: 2024-03-07
corr_author: '1'
das_tickbox: '0'
date_created: 2024-08-11T22:01:12Z
date_published: 2024-03-07T00:00:00Z
date_updated: 2026-10-01T09:28:03Z
day: '07'
ddc:
- '000'
department:
- _id: ChLa
external_id:
arxiv:
- '2306.05515'
file:
- access_level: open_access
checksum: 81b7ea2e667adaf9c7a7b6b376b1f251
content_type: application/pdf
creator: dernst
date_created: 2024-08-12T07:38:06Z
date_updated: 2024-08-12T07:38:06Z
file_id: '17415'
file_name: 2024_ICLR_Scott.pdf
file_size: 1029219
relation: main_file
success: 1
file_date_updated: 2024-08-12T07:38:06Z
has_accepted_license: '1'
language:
- iso: eng
month: '03'
oa: 1
oa_version: Published Version
publication: 12th International Conference on Learning Representations
publication_status: published
publisher: OpenReview
quality_controlled: '1'
related_material:
record:
- id: '21198'
relation: dissertation_contains
status: public
researchdata_availability: no
scopus_import: '1'
status: public
supplementarymaterial: no
title: 'PEFLL: Personalized federated learning by learning to learn'
type: conference
user_id: 317138e5-6ab7-11ef-aa6d-ffef3953e345
year: '2024'
...
---
_id: '18120'
abstract:
- lang: eng
text: In practice, training using federated learning can be orders of magnitude
slower than standard centralized training. This severely limits the amount of
experimentation and tuning that can be done, making it challenging to obtain good
performance on a given task. Server-side proxy data can be used to run training
simulations, for instance for hyperparameter tuning. This can greatly speed up
the training pipeline by reducing the number of tuning runs to be performed overall
on the true clients. However, it is challenging to ensure that these simulations
accurately reflect the dynamics of the real federated training. In particular,
the proxy data used for simulations often comes as a single centralized dataset
without a partition into distinct clients, and partitioning this data in a naive
way can lead to simulations that poorly reflect real federated training. In this
paper we address the challenge of how to partition centralized data in a way that
reflects the statistical heterogeneity of the true federated clients. We propose
a fully federated, theoretically justified, algorithm that efficiently learns
the distribution of the true clients and observe improved server-side simulations
when using the inferred distribution to create simulated clients from the centralized
data.
acknowledgement: 'We would like to thank: Mona Chitnis and everyone in the Private
Federated Learning team at Apple for their help and support throughout the entire
project; Audra McMillan, Martin Pelikan, Anosh Raj and Barry Theobold for feedback
on the initial versions of the paper; and Christoph Lampert for valuable feedback
on the paper structure and suggestions for additional experiments.'
alternative_title:
- PMLR
article_processing_charge: No
arxiv: 1
author:
- first_name: Jonathan A
full_name: Scott, Jonathan A
id: e499926b-f6e0-11ea-865d-9c63db0031e8
last_name: Scott
- first_name: Áine
full_name: Cahill, Áine
last_name: Cahill
citation:
ama: 'Scott JA, Cahill Á. Improved modelling of federated datasets using mixtures-of-Dirichlet-multinomials.
In: Proceedings of the 41st International Conference on Machine Learning.
Vol 235. ML Research Press; 2024:44012-44037.'
apa: 'Scott, J. A., & Cahill, Á. (2024). Improved modelling of federated datasets
using mixtures-of-Dirichlet-multinomials. In Proceedings of the 41st International
Conference on Machine Learning (Vol. 235, pp. 44012–44037). Vienna, Austria:
ML Research Press.'
chicago: Scott, Jonathan A, and Áine Cahill. “Improved Modelling of Federated Datasets
Using Mixtures-of-Dirichlet-Multinomials.” In Proceedings of the 41st International
Conference on Machine Learning, 235:44012–37. ML Research Press, 2024.
ieee: J. A. Scott and Á. Cahill, “Improved modelling of federated datasets using
mixtures-of-Dirichlet-multinomials,” in Proceedings of the 41st International
Conference on Machine Learning, Vienna, Austria, 2024, vol. 235, pp. 44012–44037.
ista: 'Scott JA, Cahill Á. 2024. Improved modelling of federated datasets using
mixtures-of-Dirichlet-multinomials. Proceedings of the 41st International Conference
on Machine Learning. ICML: International Conference on Machine Learning, PMLR,
vol. 235, 44012–44037.'
mla: Scott, Jonathan A., and Áine Cahill. “Improved Modelling of Federated Datasets
Using Mixtures-of-Dirichlet-Multinomials.” Proceedings of the 41st International
Conference on Machine Learning, vol. 235, ML Research Press, 2024, pp. 44012–37.
short: J.A. Scott, Á. Cahill, in:, Proceedings of the 41st International Conference
on Machine Learning, ML Research Press, 2024, pp. 44012–44037.
conference:
end_date: 2024-07-27
location: Vienna, Austria
name: 'ICML: International Conference on Machine Learning'
start_date: 2024-07-21
corr_author: '1'
das_tickbox: '0'
date_created: 2024-09-22T22:01:45Z
date_published: 2024-09-01T00:00:00Z
date_updated: 2026-10-01T09:29:04Z
day: '01'
department:
- _id: ChLa
external_id:
arxiv:
- '2406.02416'
intvolume: ' 235'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2406.02416
month: '09'
oa: 1
oa_version: Preprint
page: 44012-44037
publication: Proceedings of the 41st International Conference on Machine Learning
publication_identifier:
eissn:
- 2640-3498
publication_status: published
publisher: ML Research Press
quality_controlled: '1'
related_material:
record:
- id: '21198'
relation: dissertation_contains
status: public
researchdata_availability: no
scopus_import: '1'
status: public
supplementarymaterial: no
title: Improved modelling of federated datasets using mixtures-of-Dirichlet-multinomials
type: conference
user_id: 317138e5-6ab7-11ef-aa6d-ffef3953e345
volume: 235
year: '2024'
...
---
_id: '18118'
abstract:
- lang: eng
text: We introduce a new framework for studying meta-learning methods using PAC-Bayesian
theory. Its main advantage over previous work is that it allows for more flexibility
in how the transfer of knowledge between tasks is realized. For previous approaches,
this could only happen indirectly, by means of learning prior distributions over
models. In contrast, the new generalization bounds that we prove express the process
of meta-learning much more directly as learning the learning algorithm that should
be used for future tasks. The flexibility of our framework makes it suitable to
analyze a wide range of meta-learning mechanisms and even design new mechanisms.
Other than our theoretical contributions we also show empirically that our framework
improves the prediction quality in practical meta-learning mechanisms.
alternative_title:
- PMLR
article_processing_charge: No
arxiv: 1
author:
- first_name: Hossein
full_name: Zakerinia, Hossein
id: 653bd8b6-f394-11eb-9cf6-c0bbf6cd78d4
last_name: Zakerinia
orcid: 0009-0007-3977-6462
- first_name: Amin
full_name: Behjati, Amin
last_name: Behjati
- first_name: Christoph
full_name: Lampert, Christoph
id: 40C20FD2-F248-11E8-B48F-1D18A9856A87
last_name: Lampert
orcid: 0000-0001-8622-7887
citation:
ama: 'Zakerinia H, Behjati A, Lampert C. More flexible PAC-Bayesian meta-learning
by learning learning algorithms. In: Proceedings of the 41st International
Conference on Machine Learning. Vol 235. ML Research Press; 2024:58122-58139.'
apa: 'Zakerinia, H., Behjati, A., & Lampert, C. (2024). More flexible PAC-Bayesian
meta-learning by learning learning algorithms. In Proceedings of the 41st International
Conference on Machine Learning (Vol. 235, pp. 58122–58139). Vienna, Austria:
ML Research Press.'
chicago: Zakerinia, Hossein, Amin Behjati, and Christoph Lampert. “More Flexible
PAC-Bayesian Meta-Learning by Learning Learning Algorithms.” In Proceedings
of the 41st International Conference on Machine Learning, 235:58122–39. ML
Research Press, 2024.
ieee: H. Zakerinia, A. Behjati, and C. Lampert, “More flexible PAC-Bayesian meta-learning
by learning learning algorithms,” in Proceedings of the 41st International
Conference on Machine Learning, Vienna, Austria, 2024, vol. 235, pp. 58122–58139.
ista: 'Zakerinia H, Behjati A, Lampert C. 2024. More flexible PAC-Bayesian meta-learning
by learning learning algorithms. Proceedings of the 41st International Conference
on Machine Learning. ICML: International Conference on Machine Learning, PMLR,
vol. 235, 58122–58139.'
mla: Zakerinia, Hossein, et al. “More Flexible PAC-Bayesian Meta-Learning by Learning
Learning Algorithms.” Proceedings of the 41st International Conference on Machine
Learning, vol. 235, ML Research Press, 2024, pp. 58122–39.
short: H. Zakerinia, A. Behjati, C. Lampert, in:, Proceedings of the 41st International
Conference on Machine Learning, ML Research Press, 2024, pp. 58122–58139.
conference:
end_date: 2024-07-27
location: Vienna, Austria
name: 'ICML: International Conference on Machine Learning'
start_date: 2024-07-21
corr_author: '1'
das_tickbox: '0'
date_created: 2024-09-22T22:01:45Z
date_published: 2024-09-01T00:00:00Z
date_updated: 2026-10-01T09:28:21Z
day: '01'
ddc:
- '000'
department:
- _id: ChLa
external_id:
arxiv:
- '2402.04054'
intvolume: ' 235'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: ' https://doi.org/10.48550/arXiv.2402.04054'
month: '09'
oa: 1
oa_version: Published Version
page: 58122-58139
publication: Proceedings of the 41st International Conference on Machine Learning
publication_identifier:
eissn:
- 2640-3498
publication_status: published
publisher: ML Research Press
quality_controlled: '1'
researchdata_availability: no
scopus_import: '1'
status: public
supplementarymaterial: no
title: More flexible PAC-Bayesian meta-learning by learning learning algorithms
type: conference
user_id: 317138e5-6ab7-11ef-aa6d-ffef3953e345
volume: 235
year: '2024'
...
---
_id: '17332'
abstract:
- lang: eng
text: 'Tree data structures, such as red-black trees, quad trees, treaps, or tries,
are fundamental tools in computer science. A classical problem in concurrency
is to obtain expressive, efficient, and scalable versions of practical tree data
structures. We are interested in concurrent trees supporting range queries, i.e.,
queries that involve multiple consecutive data items. Existing implementations
with this capability can list keys in a specific range, but do not support aggregate
range queries: for instance, if we want to calculate the number of keys in a range,
the only choice is to retrieve a whole list and return its size. This is suboptimal:
in the sequential setting, one can augment a balanced search tree with counters
and, consequently, perform these aggregate requests in logarithmic rather than
linear time.In this paper, we propose a generic approach to implement a broad
class of range queries on concurrent trees in a way that is wait-free, asymptotically
efficient, and practically scalable. The key idea is a new mechanism for maintaining
metadata concurrently at tree nodes, which can be seen as a wait-free variant
of hand-over-hand locking (which we call hand-over-hand helping). We did a preliminary
implementation of the wait-free binary search tree and preliminary experiments
have indicated the soundness of our approach.'
acknowledgement: We thank Trevor Brown and Yuanhao Wei for the discussion and anonymous
reviewers for helping us to improve the paper. Also, we thank JetBrains and Huawei
for their support.
article_processing_charge: No
arxiv: 1
author:
- first_name: Ilya
full_name: Kokorin, Ilya
last_name: Kokorin
- first_name: Victor
full_name: Yudov, Victor
last_name: Yudov
- first_name: Vitaly
full_name: Aksenov, Vitaly
last_name: Aksenov
- first_name: Dan-Adrian
full_name: Alistarh, Dan-Adrian
id: 4A899BFC-F248-11E8-B48F-1D18A9856A87
last_name: Alistarh
orcid: 0000-0003-3650-940X
citation:
ama: 'Kokorin I, Yudov V, Aksenov V, Alistarh D-A. Wait-free trees with asymptotically-efficient
range queries. In: 2024 IEEE International Parallel and Distributed Processing
Symposium. IEEE; 2024:169-179. doi:10.1109/IPDPS57955.2024.00023'
apa: 'Kokorin, I., Yudov, V., Aksenov, V., & Alistarh, D.-A. (2024). Wait-free
trees with asymptotically-efficient range queries. In 2024 IEEE International
Parallel and Distributed Processing Symposium (pp. 169–179). San Francisco,
CA, United States: IEEE. https://doi.org/10.1109/IPDPS57955.2024.00023'
chicago: Kokorin, Ilya, Victor Yudov, Vitaly Aksenov, and Dan-Adrian Alistarh. “Wait-Free
Trees with Asymptotically-Efficient Range Queries.” In 2024 IEEE International
Parallel and Distributed Processing Symposium, 169–79. IEEE, 2024. https://doi.org/10.1109/IPDPS57955.2024.00023.
ieee: I. Kokorin, V. Yudov, V. Aksenov, and D.-A. Alistarh, “Wait-free trees with
asymptotically-efficient range queries,” in 2024 IEEE International Parallel
and Distributed Processing Symposium, San Francisco, CA, United States, 2024,
pp. 169–179.
ista: 'Kokorin I, Yudov V, Aksenov V, Alistarh D-A. 2024. Wait-free trees with asymptotically-efficient
range queries. 2024 IEEE International Parallel and Distributed Processing Symposium.
IPDPS: International Parallel and Distributed Processing Symposium, 169–179.'
mla: Kokorin, Ilya, et al. “Wait-Free Trees with Asymptotically-Efficient Range
Queries.” 2024 IEEE International Parallel and Distributed Processing Symposium,
IEEE, 2024, pp. 169–79, doi:10.1109/IPDPS57955.2024.00023.
short: I. Kokorin, V. Yudov, V. Aksenov, D.-A. Alistarh, in:, 2024 IEEE International
Parallel and Distributed Processing Symposium, IEEE, 2024, pp. 169–179.
conference:
end_date: 2024-05-31
location: San Francisco, CA, United States
name: 'IPDPS: International Parallel and Distributed Processing Symposium'
start_date: 2024-05-27
corr_author: '1'
das_tickbox: '0'
date_created: 2024-07-28T22:01:11Z
date_published: 2024-07-08T00:00:00Z
date_updated: 2026-10-01T09:30:18Z
day: '08'
department:
- _id: DaAl
doi: 10.1109/IPDPS57955.2024.00023
external_id:
arxiv:
- '2310.05293'
isi:
- '001270389600078'
fulldoi: https://doi.org/10.1109/IPDPS57955.2024.00023
isi: 1
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2310.05293
month: '07'
oa: 1
oa_version: Preprint
page: 169-179
publication: 2024 IEEE International Parallel and Distributed Processing Symposium
publication_identifier:
eissn:
- 1530-2075
isbn:
- '9798350337662'
publication_status: published
publisher: IEEE
quality_controlled: '1'
researchdata_availability: no
scopus_import: '1'
status: public
supplementarymaterial: no
title: Wait-free trees with asymptotically-efficient range queries
type: conference
user_id: 317138e5-6ab7-11ef-aa6d-ffef3953e345
year: '2024'
...
---
_id: '15011'
abstract:
- lang: eng
text: Pruning large language models (LLMs) from the BERT family has emerged as a
standard compression benchmark, and several pruning methods have been proposed
for this task. The recent “Sparsity May Cry” (SMC) benchmark put into question
the validity of all existing methods, exhibiting a more complex setup where many
known pruning methods appear to fail. We revisit the question of accurate BERT-pruning
during fine-tuning on downstream datasets, and propose a set of general guidelines
for successful pruning, even on the challenging SMC benchmark. First, we perform
a cost-vs-benefits analysis of pruning model components, such as the embeddings
and the classification head; second, we provide a simple-yet-general way of scaling
training, sparsification and learning rate schedules relative to the desired target
sparsity; finally, we investigate the importance of proper parametrization for
Knowledge Distillation in the context of LLMs. Our simple insights lead to state-of-the-art
results, both on classic BERT-pruning benchmarks, as well as on the SMC benchmark,
showing that even classic gradual magnitude pruning (GMP) can yield competitive
results, with the right approach.
alternative_title:
- PMLR
article_processing_charge: No
arxiv: 1
author:
- first_name: Eldar
full_name: Kurtic, Eldar
id: 47beb3a5-07b5-11eb-9b87-b108ec578218
last_name: Kurtic
- first_name: Torsten
full_name: Hoefler, Torsten
last_name: Hoefler
- first_name: Dan-Adrian
full_name: Alistarh, Dan-Adrian
id: 4A899BFC-F248-11E8-B48F-1D18A9856A87
last_name: Alistarh
orcid: 0000-0003-3650-940X
citation:
ama: 'Kurtic E, Hoefler T, Alistarh D-A. How to prune your language model: Recovering
accuracy on the “Sparsity May Cry” benchmark. In: Proceedings of Machine Learning
Research. Vol 234. ML Research Press; 2024:542-553.'
apa: 'Kurtic, E., Hoefler, T., & Alistarh, D.-A. (2024). How to prune your language
model: Recovering accuracy on the “Sparsity May Cry” benchmark. In Proceedings
of Machine Learning Research (Vol. 234, pp. 542–553). Hongkong, China: ML
Research Press.'
chicago: 'Kurtic, Eldar, Torsten Hoefler, and Dan-Adrian Alistarh. “How to Prune
Your Language Model: Recovering Accuracy on the ‘Sparsity May Cry’ Benchmark.”
In Proceedings of Machine Learning Research, 234:542–53. ML Research Press,
2024.'
ieee: 'E. Kurtic, T. Hoefler, and D.-A. Alistarh, “How to prune your language model:
Recovering accuracy on the ‘Sparsity May Cry’ benchmark,” in Proceedings of
Machine Learning Research, Hongkong, China, 2024, vol. 234, pp. 542–553.'
ista: 'Kurtic E, Hoefler T, Alistarh D-A. 2024. How to prune your language model:
Recovering accuracy on the ‘Sparsity May Cry’ benchmark. Proceedings of Machine
Learning Research. CPAL: Conference on Parsimony and Learning, PMLR, vol. 234,
542–553.'
mla: 'Kurtic, Eldar, et al. “How to Prune Your Language Model: Recovering Accuracy
on the ‘Sparsity May Cry’ Benchmark.” Proceedings of Machine Learning Research,
vol. 234, ML Research Press, 2024, pp. 542–53.'
short: E. Kurtic, T. Hoefler, D.-A. Alistarh, in:, Proceedings of Machine Learning
Research, ML Research Press, 2024, pp. 542–553.
conference:
end_date: 2024-01-06
location: Hongkong, China
name: 'CPAL: Conference on Parsimony and Learning'
start_date: 2024-01-03
corr_author: '1'
das_tickbox: '0'
date_created: 2024-02-18T23:01:03Z
date_published: 2024-01-08T00:00:00Z
date_updated: 2026-10-01T09:31:02Z
day: '08'
department:
- _id: DaAl
external_id:
arxiv:
- '2312.13547'
intvolume: ' 234'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://proceedings.mlr.press/v234/kurtic24a
month: '01'
oa: 1
oa_version: Preprint
page: 542-553
publication: Proceedings of Machine Learning Research
publication_identifier:
eissn:
- 2640-3498
publication_status: published
publisher: ML Research Press
quality_controlled: '1'
researchdata_availability: no
scopus_import: '1'
status: public
supplementarymaterial: no
title: 'How to prune your language model: Recovering accuracy on the "Sparsity May
Cry" benchmark'
type: conference
user_id: 317138e5-6ab7-11ef-aa6d-ffef3953e345
volume: 234
year: '2024'
...
---
_id: '18070'
abstract:
- lang: eng
text: Parallel SGD in a shared-memory setting is oft-represented by the popular
Hogwild! algorithm, in which lock-free updates are asynchronously performed by
multiple computing processes. Unfortunately, scaling Hogwild! to distributed workers
is largely unexplored. Specifically, it is unknown if any adaptation of Hogwild!
to the popular decentralized multi-GPU setting offers any competitive speedup,
either empirically or theoretically. In this work, we investigate the potential
of decentralizing Hogwild! by incorporating simultaneously (a) asynchronous local
gradient updates on the shared memory of GPUs, and (b) non-blocking asynchronous
decentralized federated averaging. A naive direct implementation shows degradation
in performance, arising from scheduling overheads and concurrent write conflicts
on GPUs. To mitigate these drawbacks, we investigate and propose a new method,
based on careful block selection rules, which update only portions of the parameter
vectors. Our experiments show that the resulting decentralized training method
exhibits improved throughput and competitive accuracy for standard image classification
benchmarks on the CIFAR-10, CIFAR-100, and Imagenet datasets. On the theoretical
side, we prove that our method guarantees sublinear ergodic convergence rates
for non-convex objectives.
article_processing_charge: No
author:
- first_name: Bapi
full_name: Chatterjee, Bapi
id: 3C41A08A-F248-11E8-B48F-1D18A9856A87
last_name: Chatterjee
orcid: 0000-0002-2742-4028
- first_name: Vyacheslav
full_name: Kungurtsev, Vyacheslav
last_name: Kungurtsev
- first_name: Dan-Adrian
full_name: Alistarh, Dan-Adrian
id: 4A899BFC-F248-11E8-B48F-1D18A9856A87
last_name: Alistarh
orcid: 0000-0003-3650-940X
citation:
ama: 'Chatterjee B, Kungurtsev V, Alistarh D-A. Federated SGD with local asynchrony.
In: Proceedings of the 44th International Conference on Distributed Computing
Systems. IEEE; 2024:857-868. doi:10.1109/ICDCS60910.2024.00084'
apa: 'Chatterjee, B., Kungurtsev, V., & Alistarh, D.-A. (2024). Federated SGD
with local asynchrony. In Proceedings of the 44th International Conference
on Distributed Computing Systems (pp. 857–868). Jersey City, NJ, United States:
IEEE. https://doi.org/10.1109/ICDCS60910.2024.00084'
chicago: Chatterjee, Bapi, Vyacheslav Kungurtsev, and Dan-Adrian Alistarh. “Federated
SGD with Local Asynchrony.” In Proceedings of the 44th International Conference
on Distributed Computing Systems, 857–68. IEEE, 2024. https://doi.org/10.1109/ICDCS60910.2024.00084.
ieee: B. Chatterjee, V. Kungurtsev, and D.-A. Alistarh, “Federated SGD with local
asynchrony,” in Proceedings of the 44th International Conference on Distributed
Computing Systems, Jersey City, NJ, United States, 2024, pp. 857–868.
ista: 'Chatterjee B, Kungurtsev V, Alistarh D-A. 2024. Federated SGD with local
asynchrony. Proceedings of the 44th International Conference on Distributed Computing
Systems. ICDCS: International Conference on Distributed Computing Systems, 857–868.'
mla: Chatterjee, Bapi, et al. “Federated SGD with Local Asynchrony.” Proceedings
of the 44th International Conference on Distributed Computing Systems, IEEE,
2024, pp. 857–68, doi:10.1109/ICDCS60910.2024.00084.
short: B. Chatterjee, V. Kungurtsev, D.-A. Alistarh, in:, Proceedings of the 44th
International Conference on Distributed Computing Systems, IEEE, 2024, pp. 857–868.
conference:
end_date: 2024-07-26
location: Jersey City, NJ, United States
name: 'ICDCS: International Conference on Distributed Computing Systems'
start_date: 2024-07-23
corr_author: '1'
das_tickbox: '0'
date_created: 2024-09-15T22:01:41Z
date_published: 2024-07-26T00:00:00Z
date_updated: 2026-10-01T09:32:24Z
day: '26'
department:
- _id: DaAl
doi: 10.1109/ICDCS60910.2024.00084
external_id:
isi:
- '001304430200075'
fulldoi: https://doi.org/10.1109/ICDCS60910.2024.00084
isi: 1
language:
- iso: eng
month: '07'
oa_version: None
page: 857-868
publication: Proceedings of the 44th International Conference on Distributed Computing
Systems
publication_identifier:
eissn:
- 2575-8411
isbn:
- '9798350386059'
issn:
- 1063-6927
publication_status: published
publisher: IEEE
quality_controlled: '1'
researchdata_availability: yes
scopus_import: '1'
status: public
supplementarymaterial: no
title: Federated SGD with local asynchrony
type: conference
user_id: 317138e5-6ab7-11ef-aa6d-ffef3953e345
year: '2024'
...
---
_id: '17329'
abstract:
- lang: eng
text: 'We initiate the study of game dynamics in the population protocol model:
n agents each maintain a current local strategy and interact in pairs uniformly
at random. Upon each interaction, the agents play a two-person game and receive
a payoff from an underlying utility function, and they can subsequently update
their strategies according to a fixed local algorithm. In this setting, we ask
how the distribution over agent strategies evolves over a sequence of interactions,
and we introduce a new distributional equilibrium concept to quantify the quality
of such distributions. As an initial example, we study a class of repeated prisoner''s
dilemma games, and we consider a family of simple local update algorithms that
yield non-trivial dynamics over the distribution of agent strategies. We show
that these dynamics are related to a new class of high-dimensional Ehrenfest random
walks, and we derive exact characterizations of their stationary distributions,
bounds on their mixing times, and prove their convergence to approximate distributional
equilibria. Our results highlight trade-offs between the local state space of
each agent, and the convergence rate and approximation factor of the underlying
dynamics. Our approach opens the door towards the further characterization of
equilibrium computation for other classes of games and dynamics in the population
setting.'
acknowledgement: This work was supported in part by the ERC-2020-CoG 863818 (FoRM-SMArt)
grant. We thank James Aspnes and Thomas Sauerwald for several helpful discussions
on Ehrenfest random walks.
article_processing_charge: Yes (via OA deal)
author:
- first_name: Dan-Adrian
full_name: Alistarh, Dan-Adrian
id: 4A899BFC-F248-11E8-B48F-1D18A9856A87
last_name: Alistarh
orcid: 0000-0003-3650-940X
- first_name: Krishnendu
full_name: Chatterjee, Krishnendu
id: 2E5DCA20-F248-11E8-B48F-1D18A9856A87
last_name: Chatterjee
orcid: 0000-0002-4561-241X
- first_name: Mehrdad
full_name: Karrabi, Mehrdad
id: 67638922-f394-11eb-9cf6-f20423e08757
last_name: Karrabi
orcid: 0009-0007-5253-9170
- first_name: John M
full_name: Lazarsfeld, John M
id: 17ce3656-183e-11ef-84c3-8932383e1b23
last_name: Lazarsfeld
citation:
ama: 'Alistarh D-A, Chatterjee K, Karrabi M, Lazarsfeld JM. Game dynamics and equilibrium
computation in the population protocol model. In: Proceedings of the 43rd Annual
ACM Symposium on Principles of Distributed Computing. Association for Computing
Machinery; 2024:40-49. doi:10.1145/3662158.3662768'
apa: 'Alistarh, D.-A., Chatterjee, K., Karrabi, M., & Lazarsfeld, J. M. (2024).
Game dynamics and equilibrium computation in the population protocol model. In
Proceedings of the 43rd Annual ACM Symposium on Principles of Distributed Computing
(pp. 40–49). Nantes, France: Association for Computing Machinery. https://doi.org/10.1145/3662158.3662768'
chicago: Alistarh, Dan-Adrian, Krishnendu Chatterjee, Mehrdad Karrabi, and John
M Lazarsfeld. “Game Dynamics and Equilibrium Computation in the Population Protocol
Model.” In Proceedings of the 43rd Annual ACM Symposium on Principles of Distributed
Computing, 40–49. Association for Computing Machinery, 2024. https://doi.org/10.1145/3662158.3662768.
ieee: D.-A. Alistarh, K. Chatterjee, M. Karrabi, and J. M. Lazarsfeld, “Game dynamics
and equilibrium computation in the population protocol model,” in Proceedings
of the 43rd Annual ACM Symposium on Principles of Distributed Computing, Nantes,
France, 2024, pp. 40–49.
ista: 'Alistarh D-A, Chatterjee K, Karrabi M, Lazarsfeld JM. 2024. Game dynamics
and equilibrium computation in the population protocol model. Proceedings of the
43rd Annual ACM Symposium on Principles of Distributed Computing. PODC: Symposium
on Principles of Distributed Computing, 40–49.'
mla: Alistarh, Dan-Adrian, et al. “Game Dynamics and Equilibrium Computation in
the Population Protocol Model.” Proceedings of the 43rd Annual ACM Symposium
on Principles of Distributed Computing, Association for Computing Machinery,
2024, pp. 40–49, doi:10.1145/3662158.3662768.
short: D.-A. Alistarh, K. Chatterjee, M. Karrabi, J.M. Lazarsfeld, in:, Proceedings
of the 43rd Annual ACM Symposium on Principles of Distributed Computing, Association
for Computing Machinery, 2024, pp. 40–49.
conference:
end_date: 2024-06-21
location: Nantes, France
name: 'PODC: Symposium on Principles of Distributed Computing'
start_date: 2024-06-17
corr_author: '1'
das_tickbox: '0'
date_created: 2024-07-28T22:01:10Z
date_published: 2024-06-17T00:00:00Z
date_updated: 2026-10-01T09:34:57Z
day: '17'
ddc:
- '000'
department:
- _id: DaAl
- _id: KrCh
doi: 10.1145/3662158.3662768
ec_funded: 1
file:
- access_level: open_access
checksum: 65a40437f83373fa79dd999d5287509e
content_type: application/pdf
creator: dernst
date_created: 2024-07-29T07:37:31Z
date_updated: 2024-07-29T07:37:31Z
file_id: '17335'
file_name: 2024_ACMPODC_Alistarh.pdf
file_size: 750908
relation: main_file
success: 1
file_date_updated: 2024-07-29T07:37:31Z
fulldoi: https://doi.org/10.1145/3662158.3662768
has_accepted_license: '1'
language:
- iso: eng
month: '06'
oa: 1
oa_version: Published Version
page: 40-49
project:
- _id: 0599E47C-7A3F-11EA-A408-12923DDC885E
call_identifier: H2020
grant_number: '863818'
name: 'Formal Methods for Stochastic Models: Algorithms and Applications'
publication: Proceedings of the 43rd Annual ACM Symposium on Principles of Distributed
Computing
publication_identifier:
isbn:
- '9798400706684'
publication_status: published
publisher: Association for Computing Machinery
quality_controlled: '1'
researchdata_availability: no
scopus_import: '1'
status: public
supplementarymaterial: no
title: Game dynamics and equilibrium computation in the population protocol model
tmp:
image: /images/cc_by.png
legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode
name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)
short: CC BY (4.0)
type: conference
user_id: 317138e5-6ab7-11ef-aa6d-ffef3953e345
year: '2024'
...
---
_id: '18121'
abstract:
- lang: eng
text: It is known that sparsity can improve interpretability for deep neural networks.
However, existing methods in the area either require networks that are pre-trained
with sparsity constraints, or impose sparsity after the fact, altering the network’s
general behavior. In this paper, we demonstrate, for the first time, that sparsity
can instead be incorporated into the interpretation process itself, as a sample-specific
preprocessing step. Unlike previous work, this approach, which we call SPADE,
does not place constraints on the trained model and does not affect its behavior
during inference on the sample. Given a trained model and a target sample, SPADE
uses sample-targeted pruning to provide a "trace" of the network’s execution on
the sample, reducing the network to the most important connections prior to computing
an interpretation. We demonstrate that preprocessing with SPADE significantly
increases the accuracy of image saliency maps across several interpretability
methods. Additionally, SPADE improves the usefulness of neuron visualizations,
aiding humans in reasoning about network behavior. Our code is available at https://github.com/IST-DASLab/SPADE.
acknowledged_ssus:
- _id: ScienComp
acknowledgement: The authors would like to thank Stephen Casper and Tony Wang for
their feedback on this work, and Eldar Kurtic for his advice on aspects of the project.
This research was supported by the Scientific Service Units (SSU) of IST Austria
through resources provided by Scientific Computing (SciComp). EI was supported in
part by the FWF DK VGSCO, grant agreement number W1260-N35.
alternative_title:
- PMLR
article_processing_charge: No
arxiv: 1
author:
- first_name: Arshia Soltani
full_name: Moakhar, Arshia Soltani
last_name: Moakhar
- first_name: Eugenia B
full_name: Iofinova, Eugenia B
id: f9a17499-f6e0-11ea-865d-fdf9a3f77117
last_name: Iofinova
orcid: 0000-0002-7778-3221
- first_name: Elias
full_name: Frantar, Elias
id: 09a8f98d-ec99-11ea-ae11-c063a7b7fe5f
last_name: Frantar
- first_name: Dan-Adrian
full_name: Alistarh, Dan-Adrian
id: 4A899BFC-F248-11E8-B48F-1D18A9856A87
last_name: Alistarh
orcid: 0000-0003-3650-940X
citation:
ama: 'Moakhar AS, Iofinova EB, Frantar E, Alistarh D-A. SPADE: Sparsity-guided debugging
for deep neural networks. In: Proceedings of the 41st International Conference
on Machine Learning. Vol 235. ML Research Press; 2024:45955-45987.'
apa: 'Moakhar, A. S., Iofinova, E. B., Frantar, E., & Alistarh, D.-A. (2024).
SPADE: Sparsity-guided debugging for deep neural networks. In Proceedings of
the 41st International Conference on Machine Learning (Vol. 235, pp. 45955–45987).
Vienna, Austria: ML Research Press.'
chicago: 'Moakhar, Arshia Soltani, Eugenia B Iofinova, Elias Frantar, and Dan-Adrian
Alistarh. “SPADE: Sparsity-Guided Debugging for Deep Neural Networks.” In Proceedings
of the 41st International Conference on Machine Learning, 235:45955–87. ML
Research Press, 2024.'
ieee: 'A. S. Moakhar, E. B. Iofinova, E. Frantar, and D.-A. Alistarh, “SPADE: Sparsity-guided
debugging for deep neural networks,” in Proceedings of the 41st International
Conference on Machine Learning, Vienna, Austria, 2024, vol. 235, pp. 45955–45987.'
ista: 'Moakhar AS, Iofinova EB, Frantar E, Alistarh D-A. 2024. SPADE: Sparsity-guided
debugging for deep neural networks. Proceedings of the 41st International Conference
on Machine Learning. ICML: International Conference on Machine Learning, PMLR,
vol. 235, 45955–45987.'
mla: 'Moakhar, Arshia Soltani, et al. “SPADE: Sparsity-Guided Debugging for Deep
Neural Networks.” Proceedings of the 41st International Conference on Machine
Learning, vol. 235, ML Research Press, 2024, pp. 45955–87.'
short: A.S. Moakhar, E.B. Iofinova, E. Frantar, D.-A. Alistarh, in:, Proceedings
of the 41st International Conference on Machine Learning, ML Research Press, 2024,
pp. 45955–45987.
conference:
end_date: 2024-07-27
location: Vienna, Austria
name: 'ICML: International Conference on Machine Learning'
start_date: 2024-07-21
corr_author: '1'
das_tickbox: '0'
date_created: 2024-09-22T22:01:46Z
date_published: 2024-09-01T00:00:00Z
date_updated: 2026-10-01T09:32:47Z
day: '01'
department:
- _id: DaAl
external_id:
arxiv:
- '2310.04519'
intvolume: ' 235'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2310.04519
month: '09'
oa: 1
oa_version: Preprint
page: 45955-45987
project:
- _id: 9B9290DE-BA93-11EA-9121-9846C619BF3A
grant_number: W1260-N35
name: Vienna Graduate School on Computational Optimization
publication: Proceedings of the 41st International Conference on Machine Learning
publication_identifier:
eissn:
- 2640-3498
publication_status: published
publisher: ML Research Press
quality_controlled: '1'
related_material:
link:
- relation: software
url: https://github.com/IST-DASLab/SPADE
record:
- id: '21854'
relation: dissertation_contains
status: public
researchdata_availability: yes
scopus_import: '1'
status: public
supplementarymaterial: no
title: 'SPADE: Sparsity-guided debugging for deep neural networks'
type: conference
user_id: 317138e5-6ab7-11ef-aa6d-ffef3953e345
volume: 235
year: '2024'
...
---
OA_place: repository
OA_type: green
_id: '18976'
abstract:
- lang: eng
text: We analyze asynchronous-type algorithms for distributed SGD in the heterogeneous
setting, where each worker has its own computation and communication speeds, as
well as data distribution. In these algorithms, workers compute possibly stale
and stochastic gradients associated with their local data at some iteration back
in history and then return those gradients to the server without synchronizing
with other workers. We present a unified convergence theory for non-convex smooth
functions in the heterogeneous regime. The proposed analysis provides convergence
for pure asynchronous SGD and its various modifications. Moreover, our theory
explains what affects the convergence rate and what can be done to improve the
performance of asynchronous algorithms. In particular, we introduce a novel asynchronous
method based on worker shuffling. As a by-product of our analysis, we also demonstrate
convergence guarantees for gradient-type algorithms such as SGD with random reshuffling
and shuffle-once mini-batch SGD. The derived rates match the best-known results
for those algorithms, highlighting the tightness of our approach. Finally, our
numerical evaluations support theoretical findings and show the good practical
performance of our method.
acknowledgement: "The authors thank all anonymous reviewers for their valuable comments
and suggestions on how to improve the manuscript. This work was done when Rustem
Islamov was a Master’s student at Institut Polytechnique de Paris (IP Paris) and
an intern at Institute of Science and Technology Austria (ISTA). The research of
Rustem Islamov was supported by ISTA internship\r\nprogram. Mher Safaryan has received
funding from the European Union’s Horizon 2020 research and innovation program under
the Marie Skłodowska-Curie grant agreement No 101034413."
alternative_title:
- PMLR
article_processing_charge: No
arxiv: 1
author:
- first_name: Rustem
full_name: Islamov, Rustem
last_name: Islamov
- first_name: Mher
full_name: Safaryan, Mher
id: dd546b39-0804-11ed-9c55-ef075c39778d
last_name: Safaryan
- first_name: Dan-Adrian
full_name: Alistarh, Dan-Adrian
id: 4A899BFC-F248-11E8-B48F-1D18A9856A87
last_name: Alistarh
orcid: 0000-0003-3650-940X
citation:
ama: 'Islamov R, Safaryan M, Alistarh D-A. AsGrad: A sharp unified analysis of asynchronous-SGD
algorithms. In: Proceedings of The 27th International Conference on Artificial
Intelligence and Statistics. Vol 238. ML Research Press; 2024:649-657.'
apa: 'Islamov, R., Safaryan, M., & Alistarh, D.-A. (2024). AsGrad: A sharp unified
analysis of asynchronous-SGD algorithms. In Proceedings of The 27th International
Conference on Artificial Intelligence and Statistics (Vol. 238, pp. 649–657).
Valencia, Spain: ML Research Press.'
chicago: 'Islamov, Rustem, Mher Safaryan, and Dan-Adrian Alistarh. “AsGrad: A Sharp
Unified Analysis of Asynchronous-SGD Algorithms.” In Proceedings of The 27th
International Conference on Artificial Intelligence and Statistics, 238:649–57.
ML Research Press, 2024.'
ieee: 'R. Islamov, M. Safaryan, and D.-A. Alistarh, “AsGrad: A sharp unified analysis
of asynchronous-SGD algorithms,” in Proceedings of The 27th International Conference
on Artificial Intelligence and Statistics, Valencia, Spain, 2024, vol. 238,
pp. 649–657.'
ista: 'Islamov R, Safaryan M, Alistarh D-A. 2024. AsGrad: A sharp unified analysis
of asynchronous-SGD algorithms. Proceedings of The 27th International Conference
on Artificial Intelligence and Statistics. AISTATS: Conference on Artificial Intelligence
and Statistics, PMLR, vol. 238, 649–657.'
mla: 'Islamov, Rustem, et al. “AsGrad: A Sharp Unified Analysis of Asynchronous-SGD
Algorithms.” Proceedings of The 27th International Conference on Artificial
Intelligence and Statistics, vol. 238, ML Research Press, 2024, pp. 649–57.'
short: R. Islamov, M. Safaryan, D.-A. Alistarh, in:, Proceedings of The 27th International
Conference on Artificial Intelligence and Statistics, ML Research Press, 2024,
pp. 649–657.
conference:
end_date: 2024-05-04
location: Valencia, Spain
name: 'AISTATS: Conference on Artificial Intelligence and Statistics'
start_date: 2024-05-02
corr_author: '1'
das_tickbox: '0'
date_created: 2025-01-30T08:15:49Z
date_published: 2024-05-15T00:00:00Z
date_updated: 2026-10-01T09:33:24Z
day: '15'
department:
- _id: DaAl
ec_funded: 1
external_id:
arxiv:
- '2310.20452'
intvolume: ' 238'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2310.20452
month: '05'
oa: 1
oa_version: Preprint
page: 649-657
project:
- _id: fc2ed2f7-9c52-11eb-aca3-c01059dda49c
call_identifier: H2020
grant_number: '101034413'
name: 'IST-BRIDGE: International postdoctoral program'
publication: Proceedings of The 27th International Conference on Artificial Intelligence
and Statistics
publication_identifier:
eissn:
- 2640-3498
publication_status: published
publisher: ML Research Press
quality_controlled: '1'
researchdata_availability: no
scopus_import: '1'
status: public
supplementarymaterial: no
title: 'AsGrad: A sharp unified analysis of asynchronous-SGD algorithms'
type: conference
user_id: 317138e5-6ab7-11ef-aa6d-ffef3953e345
volume: 238
year: '2024'
...
---
_id: '18113'
abstract:
- lang: eng
text: 'The emergence of accurate open large language models (LLMs) has led to a
race towards performant quantization techniques which can enable their execution
on end-user devices. In this paper, we revisit the problem of “extreme” LLM compression—defined
as targeting extremely low bit counts, such as 2 to 3 bits per parameter—from
the point of view of classic methods in Multi-Codebook Quantization (MCQ). Our
algorithm, called AQLM, generalizes the classic Additive Quantization (AQ) approach
for information retrieval to advance the state-of-the-art in LLM compression,
via two innovations: 1) learned additive quantization of weight matrices in input-adaptive
fashion, and 2) joint optimization of codebook parameters across each transformer
blocks. Broadly, AQLM is the first scheme that is Pareto optimal in terms of accuracy-vs-model-size
when compressing to less than 3 bits per parameter, and significantly improves
upon all known schemes in the extreme compression (2bit) regime. In addition,
AQLM is practical: we provide fast GPU and CPU implementations of AQLM for token
generation, which enable us to match or outperform optimized FP16 implementations
for speed, while executing in a much smaller memory footprint.'
acknowledgement: "Authors would like to thank Ruslan Svirschevski for his help in
solving technical issues with AQLM and baselines. We also thank Tim Dettmers for
helpful discussions on the structure of weights in modern LLMs and size-accuracy
trade-offs. The authors would also like to thank Daniil Pavlov for his assistance
with CPU benchmarking. Finally, authors would like to thank the communities of ML
enthusiasts known as LocalLLaMA5 and Petals community on discord6\r\nfor the crowd
wisdom about running LLMs on consumer devices. Egiazarian Vage and Denis Kuznedelev
and Andrei Panferov were supported by the grant for research centers in the field
of AI provided by the Analytical Center for the Government of the Russian Federation
(ACRF) in\r\naccordance with the agreement on the provision of subsidies (identifier
of the agreement 000000D730321P5Q0002) and the agreement with HSE University No.
70-2021-00139."
alternative_title:
- PMLR
article_processing_charge: No
arxiv: 1
author:
- first_name: Vage
full_name: Egiazarian, Vage
last_name: Egiazarian
- first_name: Andrei
full_name: Panferov, Andrei
id: 2c18daae-4dbe-11ef-8491-98ce2d960f09
last_name: Panferov
- first_name: Denis
full_name: Kuznedelev, Denis
last_name: Kuznedelev
- first_name: Elias
full_name: Frantar, Elias
id: 09a8f98d-ec99-11ea-ae11-c063a7b7fe5f
last_name: Frantar
- first_name: Artem
full_name: Babenko, Artem
last_name: Babenko
- first_name: Dan-Adrian
full_name: Alistarh, Dan-Adrian
id: 4A899BFC-F248-11E8-B48F-1D18A9856A87
last_name: Alistarh
orcid: 0000-0003-3650-940X
citation:
ama: 'Egiazarian V, Panferov A, Kuznedelev D, Frantar E, Babenko A, Alistarh D-A.
Extreme compression of large language models via additive quantization. In: Proceedings
of the 41st International Conference on Machine Learning. Vol 235. ML Research
Press; 2024:12284-12303.'
apa: 'Egiazarian, V., Panferov, A., Kuznedelev, D., Frantar, E., Babenko, A., &
Alistarh, D.-A. (2024). Extreme compression of large language models via additive
quantization. In Proceedings of the 41st International Conference on Machine
Learning (Vol. 235, pp. 12284–12303). Vienna, Austria: ML Research Press.'
chicago: Egiazarian, Vage, Andrei Panferov, Denis Kuznedelev, Elias Frantar, Artem
Babenko, and Dan-Adrian Alistarh. “Extreme Compression of Large Language Models
via Additive Quantization.” In Proceedings of the 41st International Conference
on Machine Learning, 235:12284–303. ML Research Press, 2024.
ieee: V. Egiazarian, A. Panferov, D. Kuznedelev, E. Frantar, A. Babenko, and D.-A.
Alistarh, “Extreme compression of large language models via additive quantization,”
in Proceedings of the 41st International Conference on Machine Learning,
Vienna, Austria, 2024, vol. 235, pp. 12284–12303.
ista: 'Egiazarian V, Panferov A, Kuznedelev D, Frantar E, Babenko A, Alistarh D-A.
2024. Extreme compression of large language models via additive quantization.
Proceedings of the 41st International Conference on Machine Learning. ICML: International
Conference on Machine Learning, PMLR, vol. 235, 12284–12303.'
mla: Egiazarian, Vage, et al. “Extreme Compression of Large Language Models via
Additive Quantization.” Proceedings of the 41st International Conference on
Machine Learning, vol. 235, ML Research Press, 2024, pp. 12284–303.
short: V. Egiazarian, A. Panferov, D. Kuznedelev, E. Frantar, A. Babenko, D.-A.
Alistarh, in:, Proceedings of the 41st International Conference on Machine Learning,
ML Research Press, 2024, pp. 12284–12303.
conference:
end_date: 2024-07-27
location: Vienna, Austria
name: 'ICML: International Conference on Machine Learning'
start_date: 2024-07-21
corr_author: '1'
das_tickbox: '0'
date_created: 2024-09-22T22:01:43Z
date_published: 2024-09-01T00:00:00Z
date_updated: 2026-10-01T09:34:19Z
day: '01'
department:
- _id: DaAl
- _id: GradSch
external_id:
arxiv:
- '2401.06118'
intvolume: ' 235'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: ' https://doi.org/10.48550/arXiv.2401.06118'
month: '09'
oa: 1
oa_version: Preprint
page: 12284-12303
publication: Proceedings of the 41st International Conference on Machine Learning
publication_identifier:
eissn:
- 2640-3498
publication_status: published
publisher: ML Research Press
quality_controlled: '1'
researchdata_availability: no
scopus_import: '1'
status: public
supplementarymaterial: no
title: Extreme compression of large language models via additive quantization
type: conference
user_id: 317138e5-6ab7-11ef-aa6d-ffef3953e345
volume: 235
year: '2024'
...
---
OA_place: repository
OA_type: green
_id: '18975'
abstract:
- lang: eng
text: Leveraging second-order information about the loss at the scale of deep networks
is one of the main lines of approach for improving the performance of current
optimizers for deep learning. Yet, existing approaches for accurate full-matrix
preconditioning, such as Full-Matrix Adagrad (GGT) or Matrix-Free Approximate
Curvature (M-FAC) suffer from massive storage costs when applied even to small-scale
models, as they must store a sliding window of gradients, whose memory requirements
are multiplicative in the model dimension. In this paper, we address this issue
via a novel and efficient error-feedback technique that can be applied to compress
preconditioners by up to two orders of magnitude in practice, without loss of
convergence. Specifically, our approach compresses the gradient information via
sparsification or low-rank compression before it is fed into the preconditioner,
feeding the compression error back into future iterations. Extensive experiments
on deep neural networks show that this approach can compress full-matrix preconditioners
to up to 99% sparsity without accuracy loss, effectively removing the memory overhead
of fullmatrix preconditioners such as GGT and M-FAC.
acknowledged_ssus:
- _id: CampIT
acknowledgement: The authors thank Adrian Vladu, Razvan Pascanu, Alexandra Peste,
Mher Safaryan for their valuable feedback, the IT department from Institute of Science
and Technology Austria for the hardware support and Weights and Biases for the infrastructure
to track all our experiments.
alternative_title:
- PMLR
article_processing_charge: No
arxiv: 1
author:
- first_name: Ionut-Vlad
full_name: Modoranu, Ionut-Vlad
id: 449f7a18-f128-11eb-9611-9b430c0c6333
last_name: Modoranu
- first_name: Aleksei
full_name: Kalinov, Aleksei
id: 44b7120e-eb97-11eb-a6c2-e1557aa81d02
last_name: Kalinov
orcid: 0000-0003-2189-3904
- first_name: Eldar
full_name: Kurtic, Eldar
id: 47beb3a5-07b5-11eb-9b87-b108ec578218
last_name: Kurtic
- first_name: Elias
full_name: Frantar, Elias
id: 09a8f98d-ec99-11ea-ae11-c063a7b7fe5f
last_name: Frantar
- first_name: Dan-Adrian
full_name: Alistarh, Dan-Adrian
id: 4A899BFC-F248-11E8-B48F-1D18A9856A87
last_name: Alistarh
orcid: 0000-0003-3650-940X
citation:
ama: 'Modoranu I-V, Kalinov A, Kurtic E, Frantar E, Alistarh D-A. Error feedback
can accurately compress preconditioners. In: 41st International Conference
on Machine Learning. Vol 235. ML Research Press; 2024:35910-35933.'
apa: 'Modoranu, I.-V., Kalinov, A., Kurtic, E., Frantar, E., & Alistarh, D.-A.
(2024). Error feedback can accurately compress preconditioners. In 41st International
Conference on Machine Learning (Vol. 235, pp. 35910–35933). Vienna, Austria:
ML Research Press.'
chicago: Modoranu, Ionut-Vlad, Aleksei Kalinov, Eldar Kurtic, Elias Frantar, and
Dan-Adrian Alistarh. “Error Feedback Can Accurately Compress Preconditioners.”
In 41st International Conference on Machine Learning, 235:35910–33. ML
Research Press, 2024.
ieee: I.-V. Modoranu, A. Kalinov, E. Kurtic, E. Frantar, and D.-A. Alistarh, “Error
feedback can accurately compress preconditioners,” in 41st International Conference
on Machine Learning, Vienna, Austria, 2024, vol. 235, pp. 35910–35933.
ista: 'Modoranu I-V, Kalinov A, Kurtic E, Frantar E, Alistarh D-A. 2024. Error feedback
can accurately compress preconditioners. 41st International Conference on Machine
Learning. ICML: International Conference on Machine Learning, PMLR, vol. 235,
35910–35933.'
mla: Modoranu, Ionut-Vlad, et al. “Error Feedback Can Accurately Compress Preconditioners.”
41st International Conference on Machine Learning, vol. 235, ML Research
Press, 2024, pp. 35910–33.
short: I.-V. Modoranu, A. Kalinov, E. Kurtic, E. Frantar, D.-A. Alistarh, in:, 41st
International Conference on Machine Learning, ML Research Press, 2024, pp. 35910–35933.
conference:
end_date: 2024-07-27
location: Vienna, Austria
name: 'ICML: International Conference on Machine Learning'
start_date: 2024-07-21
corr_author: '1'
das_tickbox: '0'
date_created: 2025-01-30T07:53:22Z
date_published: 2024-07-30T00:00:00Z
date_updated: 2026-10-01T09:33:07Z
day: '30'
department:
- _id: DaAl
external_id:
arxiv:
- '2306.06098'
intvolume: ' 235'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2306.06098
month: '07'
oa: 1
oa_version: Preprint
page: 35910-35933
publication: 41st International Conference on Machine Learning
publication_identifier:
eissn:
- 2640-3498
publication_status: published
publisher: ML Research Press
quality_controlled: '1'
researchdata_availability: yes
scopus_import: '1'
status: public
supplementarymaterial: no
title: Error feedback can accurately compress preconditioners
type: conference
user_id: 317138e5-6ab7-11ef-aa6d-ffef3953e345
volume: 235
year: '2024'
...
---
OA_place: repository
OA_type: green
_id: '18977'
abstract:
- lang: eng
text: "Recent advances in large language model (LLM) pretraining have led to high-quality
LLMs with impressive abilities. By compressing such LLMs via quantization to 3-4
bits per parameter, they can fit into memory-limited devices such as laptops and
mobile phones, enabling personalized use. Quantizing models to 3-4 bits per parameter
can lead to moderate to high accuracy losses, especially for smaller models (1-10B
parameters), which are suitable for edge deployment. To address this accuracy
issue, we introduce the Sparse-Quantized Representation (SpQR), a new compressed
format and quantization technique that enables for the first time \\emph{near-lossless}
compression of LLMs across model scales while reaching similar compression levels
to previous methods. SpQR works by identifying and isolating \\emph{outlier weights},
which cause particularly large quantization errors, and storing them in higher
precision while compressing all other weights to 3-4 bits, and achieves relative
accuracy losses of less than \r\n in perplexity for highly-accurate LLaMA and
Falcon LLMs. This makes it possible to run a 33B parameter LLM on a single 24
GB consumer GPU without performance degradation at 15% speedup, thus making powerful
LLMs available to consumers without any downsides. SpQR comes with efficient algorithms
for both encoding weights into its format, as well as decoding them efficiently
at runtime. Specifically, we provide an efficient GPU inference algorithm for
SpQR, which yields faster inference than 16-bit baselines at similar accuracy
while enabling memory compression gains of more than 4x."
acknowledgement: "Denis Kuznedelev acknowledges the support from the Russian Ministry
of Science and Higher\r\nEducation, grant No. 075-10-2021-068. Ruslan Svirschevski
and Vage Egiazarian and Denis\r\nKuznedelev were supported by the grant for research
centers in the field of AI provided by the\r\nAnalytical Center for the Government
of the Russian Federation (ACRF) in accordance with the\r\nagreement on the provision
of subsidies (identifier of the agreement 000000D730321P5Q0002) and the agreement
with HSE University No. 70-2021-00139."
article_processing_charge: No
arxiv: 1
author:
- first_name: Tim
full_name: Dettmers, Tim
last_name: Dettmers
- first_name: Ruslan A.
full_name: Svirschevski, Ruslan A.
last_name: Svirschevski
- first_name: Vage
full_name: Egiazarian, Vage
last_name: Egiazarian
- first_name: Denis
full_name: Kuznedelev, Denis
last_name: Kuznedelev
- first_name: Elias
full_name: Frantar, Elias
id: 09a8f98d-ec99-11ea-ae11-c063a7b7fe5f
last_name: Frantar
- first_name: Saleh
full_name: Ashkboos, Saleh
last_name: Ashkboos
- first_name: Alexander
full_name: Borzunov, Alexander
last_name: Borzunov
- first_name: Torsten
full_name: Hoefler, Torsten
last_name: Hoefler
- first_name: Dan-Adrian
full_name: Alistarh, Dan-Adrian
id: 4A899BFC-F248-11E8-B48F-1D18A9856A87
last_name: Alistarh
orcid: 0000-0003-3650-940X
citation:
ama: 'Dettmers T, Svirschevski RA, Egiazarian V, et al. SpQR: A sparse-quantized
representation for near-lossless LLM weight compression. In: 12th International
Conference on Learning Representations. OpenReview; 2024.'
apa: 'Dettmers, T., Svirschevski, R. A., Egiazarian, V., Kuznedelev, D., Frantar,
E., Ashkboos, S., … Alistarh, D.-A. (2024). SpQR: A sparse-quantized representation
for near-lossless LLM weight compression. In 12th International Conference
on Learning Representations. Vienna, Austria: OpenReview.'
chicago: 'Dettmers, Tim, Ruslan A. Svirschevski, Vage Egiazarian, Denis Kuznedelev,
Elias Frantar, Saleh Ashkboos, Alexander Borzunov, Torsten Hoefler, and Dan-Adrian
Alistarh. “SpQR: A Sparse-Quantized Representation for near-Lossless LLM Weight
Compression.” In 12th International Conference on Learning Representations.
OpenReview, 2024.'
ieee: 'T. Dettmers et al., “SpQR: A sparse-quantized representation for near-lossless
LLM weight compression,” in 12th International Conference on Learning Representations,
Vienna, Austria, 2024.'
ista: 'Dettmers T, Svirschevski RA, Egiazarian V, Kuznedelev D, Frantar E, Ashkboos
S, Borzunov A, Hoefler T, Alistarh D-A. 2024. SpQR: A sparse-quantized representation
for near-lossless LLM weight compression. 12th International Conference on Learning
Representations. ICLR: International Conference on Learning Representations.'
mla: 'Dettmers, Tim, et al. “SpQR: A Sparse-Quantized Representation for near-Lossless
LLM Weight Compression.” 12th International Conference on Learning Representations,
OpenReview, 2024.'
short: T. Dettmers, R.A. Svirschevski, V. Egiazarian, D. Kuznedelev, E. Frantar,
S. Ashkboos, A. Borzunov, T. Hoefler, D.-A. Alistarh, in:, 12th International
Conference on Learning Representations, OpenReview, 2024.
conference:
end_date: 2024-05-11
location: Vienna, Austria
name: 'ICLR: International Conference on Learning Representations'
start_date: 2024-05-07
das_tickbox: '0'
date_created: 2025-01-30T08:26:59Z
date_published: 2024-05-15T00:00:00Z
date_updated: 2026-10-01T09:33:44Z
day: '15'
department:
- _id: DaAl
external_id:
arxiv:
- '2306.03078'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2306.03078
month: '05'
oa: 1
oa_version: Preprint
publication: 12th International Conference on Learning Representations
publication_status: published
publisher: OpenReview
quality_controlled: '1'
researchdata_availability: no
scopus_import: '1'
status: public
supplementarymaterial: no
title: 'SpQR: A sparse-quantized representation for near-lossless LLM weight compression'
type: conference
user_id: 317138e5-6ab7-11ef-aa6d-ffef3953e345
year: '2024'
...
---
_id: '17093'
abstract:
- lang: eng
text: 'Federated Learning (FL) enables large-scale distributed training of machine
learning models, while still allowing individual nodes to maintain data locally.
However, executing FL at scale comes with inherent practical challenges: 1) heterogeneity
of the local node data distributions, 2) heterogeneity of node computational speeds
(asynchrony), but also 3) constraints in the amount of communication between the
clients and the server. In this work, we present the first variant of the classic
federated averaging (FedAvg) algorithm which, at the same time, supports data
heterogeneity, partial client asynchrony, and communication compression. Our algorithm
comes with a novel, rigorous analysis showing that, in spite of these system relaxations,
it can provide similar convergence to FedAvg in interesting parameter regimes.
Experimental results in the rigorous LEAF benchmark on setups of up to 300 nodes
show that our algorithm ensures fast convergence for standard federated tasks,
improving upon prior quantized and asynchronous approaches.'
alternative_title:
- PMLR
article_processing_charge: No
arxiv: 1
author:
- first_name: Hossein
full_name: Zakerinia, Hossein
id: 653bd8b6-f394-11eb-9cf6-c0bbf6cd78d4
last_name: Zakerinia
orcid: 0009-0007-3977-6462
- first_name: Shayan
full_name: Talaei, Shayan
last_name: Talaei
- first_name: Giorgi
full_name: Nadiradze, Giorgi
id: 3279A00C-F248-11E8-B48F-1D18A9856A87
last_name: Nadiradze
orcid: 0000-0001-5634-0731
- first_name: Dan-Adrian
full_name: Alistarh, Dan-Adrian
id: 4A899BFC-F248-11E8-B48F-1D18A9856A87
last_name: Alistarh
orcid: 0000-0003-3650-940X
citation:
ama: 'Zakerinia H, Talaei S, Nadiradze G, Alistarh D-A. Communication-efficient
federated learning with data and client heterogeneity. In: Proceedings of the
27th International Conference on Artificial Intelligence and Statistics. Vol
238. ML Research Press; 2024:3448-3456.'
apa: 'Zakerinia, H., Talaei, S., Nadiradze, G., & Alistarh, D.-A. (2024). Communication-efficient
federated learning with data and client heterogeneity. In Proceedings of the
27th International Conference on Artificial Intelligence and Statistics (Vol.
238, pp. 3448–3456). Valencia, Spain: ML Research Press.'
chicago: Zakerinia, Hossein, Shayan Talaei, Giorgi Nadiradze, and Dan-Adrian Alistarh.
“Communication-Efficient Federated Learning with Data and Client Heterogeneity.”
In Proceedings of the 27th International Conference on Artificial Intelligence
and Statistics, 238:3448–56. ML Research Press, 2024.
ieee: H. Zakerinia, S. Talaei, G. Nadiradze, and D.-A. Alistarh, “Communication-efficient
federated learning with data and client heterogeneity,” in Proceedings of the
27th International Conference on Artificial Intelligence and Statistics, Valencia,
Spain, 2024, vol. 238, pp. 3448–3456.
ista: 'Zakerinia H, Talaei S, Nadiradze G, Alistarh D-A. 2024. Communication-efficient
federated learning with data and client heterogeneity. Proceedings of the 27th
International Conference on Artificial Intelligence and Statistics. AISTATS: Conference
on Artificial Intelligence and Statistics, PMLR, vol. 238, 3448–3456.'
mla: Zakerinia, Hossein, et al. “Communication-Efficient Federated Learning with
Data and Client Heterogeneity.” Proceedings of the 27th International Conference
on Artificial Intelligence and Statistics, vol. 238, ML Research Press, 2024,
pp. 3448–56.
short: H. Zakerinia, S. Talaei, G. Nadiradze, D.-A. Alistarh, in:, Proceedings of
the 27th International Conference on Artificial Intelligence and Statistics, ML
Research Press, 2024, pp. 3448–3456.
conference:
end_date: 2024-05-04
location: Valencia, Spain
name: 'AISTATS: Conference on Artificial Intelligence and Statistics'
start_date: 2024-05-02
corr_author: '1'
das_tickbox: '0'
date_created: 2024-06-02T22:00:57Z
date_published: 2024-05-01T00:00:00Z
date_updated: 2026-10-01T09:34:02Z
day: '01'
department:
- _id: DaAl
- _id: ChLa
external_id:
arxiv:
- '2206.10032'
intvolume: ' 238'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2206.10032
month: '05'
oa: 1
oa_version: Preprint
page: 3448-3456
publication: Proceedings of the 27th International Conference on Artificial Intelligence
and Statistics
publication_identifier:
eissn:
- 2640-3498
publication_status: published
publisher: ML Research Press
quality_controlled: '1'
researchdata_availability: no
scopus_import: '1'
status: public
supplementarymaterial: no
title: Communication-efficient federated learning with data and client heterogeneity
type: conference
user_id: 317138e5-6ab7-11ef-aa6d-ffef3953e345
volume: 238
year: '2024'
...
---
_id: '18117'
abstract:
- lang: eng
text: "We investigate parameter-efficient fine-tuning (PEFT) methods that can provide
good accuracy under limited computational and memory budgets in the context of
large language models (LLMs). We present a new PEFT method called Robust Adaptation
(RoSA) inspired by robust principal component analysis that jointly trains low-rank\r\n
and highly-sparse components on top of a set of fixed pretrained weights to efficiently
approximate the performance of a full-fine-tuning (FFT) solution. Across a series
of challenging generative tasks such as grade-school math and SQL query generation,
which require fine-tuning for good performance, we show that RoSA outperforms
LoRA, pure sparse fine-tuning, and alternative hybrid methods at the same parameter
budget, and can even recover the performance of FFT on some tasks. We provide
system support for RoSA to complement the training algorithm, specifically in
the form of sparse GPU kernels which enable memory- and computationally-efficient
training, and show that it is also compatible with low-precision base weights,
resulting in the first joint representation combining quantization, low-rank and
sparse approximations. Our code is available at https://github.com/IST-DASLab/RoSA."
acknowledgement: The authors would like to thank Eldar Kurtic for experimental support
and useful suggestions throughout the project
article_processing_charge: No
arxiv: 1
author:
- first_name: Mahdi
full_name: Nikdan, Mahdi
id: 66374281-f394-11eb-9cf6-869147deecc0
last_name: Nikdan
- first_name: Soroush
full_name: Tabesh, Soroush
id: 06000900-6068-11ef-8d61-c2472ef2e752
last_name: Tabesh
orcid: 0009-0003-4119-6281
- first_name: Elvir
full_name: Crncevic, Elvir
id: 41888001-440d-11ef-8299-d0e838b8185e
last_name: Crncevic
- first_name: Dan-Adrian
full_name: Alistarh, Dan-Adrian
id: 4A899BFC-F248-11E8-B48F-1D18A9856A87
last_name: Alistarh
orcid: 0000-0003-3650-940X
citation:
ama: 'Nikdan M, Tabesh S, Crncevic E, Alistarh D-A. RoSA: Accurate parameter-efficient
fine-tuning via robust adaptation. In: Proceedings of the 41st International
Conference on Machine Learning. Vol 235. ML Research Press; 2024:38187-38206.'
apa: 'Nikdan, M., Tabesh, S., Crncevic, E., & Alistarh, D.-A. (2024). RoSA:
Accurate parameter-efficient fine-tuning via robust adaptation. In Proceedings
of the 41st International Conference on Machine Learning (Vol. 235, pp. 38187–38206).
Vienna, Austria: ML Research Press.'
chicago: 'Nikdan, Mahdi, Soroush Tabesh, Elvir Crncevic, and Dan-Adrian Alistarh.
“RoSA: Accurate Parameter-Efficient Fine-Tuning via Robust Adaptation.” In Proceedings
of the 41st International Conference on Machine Learning, 235:38187–206. ML
Research Press, 2024.'
ieee: 'M. Nikdan, S. Tabesh, E. Crncevic, and D.-A. Alistarh, “RoSA: Accurate parameter-efficient
fine-tuning via robust adaptation,” in Proceedings of the 41st International
Conference on Machine Learning, Vienna, Austria, 2024, vol. 235, pp. 38187–38206.'
ista: 'Nikdan M, Tabesh S, Crncevic E, Alistarh D-A. 2024. RoSA: Accurate parameter-efficient
fine-tuning via robust adaptation. Proceedings of the 41st International Conference
on Machine Learning. ICML: International Conference on Machine Learning vol. 235,
38187–38206.'
mla: 'Nikdan, Mahdi, et al. “RoSA: Accurate Parameter-Efficient Fine-Tuning via
Robust Adaptation.” Proceedings of the 41st International Conference on Machine
Learning, vol. 235, ML Research Press, 2024, pp. 38187–206.'
short: M. Nikdan, S. Tabesh, E. Crncevic, D.-A. Alistarh, in:, Proceedings of the
41st International Conference on Machine Learning, ML Research Press, 2024, pp.
38187–38206.
conference:
end_date: 2024-07-27
location: Vienna, Austria
name: 'ICML: International Conference on Machine Learning'
start_date: 2024-07-21
corr_author: '1'
das_tickbox: '0'
date_created: 2024-09-22T22:01:44Z
date_published: 2024-09-01T00:00:00Z
date_updated: 2026-10-01T09:34:36Z
day: '01'
department:
- _id: DaAl
- _id: GradSch
external_id:
arxiv:
- '2401.04679'
intvolume: ' 235'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2401.04679
month: '09'
oa: 1
oa_version: Preprint
page: 38187-38206
publication: Proceedings of the 41st International Conference on Machine Learning
publication_identifier:
eissn:
- 2640-3498
publication_status: published
publisher: ML Research Press
quality_controlled: '1'
related_material:
link:
- relation: software
url: https://github.com/IST-DASLab/RoSA
researchdata_availability: yes
scopus_import: '1'
status: public
supplementarymaterial: no
title: 'RoSA: Accurate parameter-efficient fine-tuning via robust adaptation'
type: conference
user_id: 317138e5-6ab7-11ef-aa6d-ffef3953e345
volume: 235
year: '2024'
...
---
_id: '15315'
abstract:
- lang: eng
text: Single and collective cell migration are fundamental processes critical for
physiological phenomena ranging from embryonic development and immune response
to wound healing and cancer metastasis. To understand cell migration from a physical
perspective, a broad variety of models for the underlying physical mechanisms
that govern cell motility have been developed. A key challenge in the development
of such models is how to connect them to experimental observations, which often
exhibit complex stochastic behaviours. In this review, we discuss recent advances
in data-driven theoretical approaches that directly connect with experimental
data to infer dynamical models of stochastic cell migration. Leveraging advances
in nanofabrication, image analysis, and tracking technology, experimental studies
now provide unprecedented large datasets on cellular dynamics. In parallel, theoretical
efforts have been directed towards integrating such datasets into physical models
from the single cell to the tissue scale with the aim of conceptualising the emergent
behaviour of cells. We first review how this inference problem has been addressed
in both freely migrating and confined cells. Next, we discuss why these dynamics
typically take the form of underdamped stochastic equations of motion, and how
such equations can be inferred from data. We then review applications of data-driven
inference and machine learning approaches to heterogeneity in cell behaviour,
subcellular degrees of freedom, and to the collective dynamics of multicellular
systems. Across these applications, we emphasise how data-driven methods can be
integrated with physical active matter models of migrating cells, and help reveal
how underlying molecular mechanisms control cell behaviour. Together, these data-driven
approaches are a promising avenue for building physical models of cell migration
directly from experimental data, and for providing conceptual links between different
length-scales of description.
acknowledgement: This work was supported by the Deutsche Forschungsgemeinschaft (German
Research Foundation)—Project-ID 201269156—SFB 1032 (Project B12). D B B was supported
by an NOMIS Fellowship and an EMBO Fellowship (ALTF 343-2022). We thank Joachim
Rädler, Alexandra Fink, Erwin Frey, Pierre Ronceray, Ricard Alert, Edouard Hannezo,
Henrik Flyvbjerg, Ulrich Schwarz, Joshua Shaevitz, Greg Stephens, Andrea Cavagna,
Grzegorz Gradziuk, Fridtjof Brauns, Nikolas Claussen, Tom Brandstätter, Johannes
Flommersfeld, Christoph Schreiber, Nicolas Arlt, Matthew Schmitt, Joris Messelink,
Federico Gnesotto, Federica Mura, Bram Hoogland, Manon Wigbers, Isabella Graf, Jessica
Lober, and many others for inspiring discussions. We also thank Claudia Flandoli
for the artwork in figures 1, 5, 8 and 9.
article_number: '056601'
article_processing_charge: Yes (in subscription journal)
article_type: review
arxiv: 1
author:
- first_name: David
full_name: Brückner, David
id: e1e86031-6537-11eb-953a-f7ab92be508d
last_name: Brückner
orcid: 0000-0001-7205-2975
- first_name: Chase P.
full_name: Broedersz, Chase P.
last_name: Broedersz
citation:
ama: 'Brückner D, Broedersz CP. Learning dynamical models of single and collective
cell migration: a review. Reports on Progress in Physics. 2024;87(5). doi:10.1088/1361-6633/ad36d2'
apa: 'Brückner, D., & Broedersz, C. P. (2024). Learning dynamical models of
single and collective cell migration: a review. Reports on Progress in Physics.
IOP Publishing. https://doi.org/10.1088/1361-6633/ad36d2'
chicago: 'Brückner, David, and Chase P. Broedersz. “Learning Dynamical Models of
Single and Collective Cell Migration: A Review.” Reports on Progress in Physics.
IOP Publishing, 2024. https://doi.org/10.1088/1361-6633/ad36d2.'
ieee: 'D. Brückner and C. P. Broedersz, “Learning dynamical models of single and
collective cell migration: a review,” Reports on Progress in Physics, vol.
87, no. 5. IOP Publishing, 2024.'
ista: 'Brückner D, Broedersz CP. 2024. Learning dynamical models of single and collective
cell migration: a review. Reports on Progress in Physics. 87(5), 056601.'
mla: 'Brückner, David, and Chase P. Broedersz. “Learning Dynamical Models of Single
and Collective Cell Migration: A Review.” Reports on Progress in Physics,
vol. 87, no. 5, 056601, IOP Publishing, 2024, doi:10.1088/1361-6633/ad36d2.'
short: D. Brückner, C.P. Broedersz, Reports on Progress in Physics 87 (2024).
corr_author: '1'
das_tickbox: '1'
dataavailabilitystatement: No new data were created or analysed in this study.
date_created: 2024-04-14T22:01:01Z
date_published: 2024-04-04T00:00:00Z
date_updated: 2026-10-01T10:09:41Z
day: '04'
ddc:
- '530'
department:
- _id: EdHa
doi: 10.1088/1361-6633/ad36d2
external_id:
arxiv:
- '2309.00545'
isi:
- '001196692400001'
pmid:
- '38518358'
file:
- access_level: open_access
checksum: c5910078230ade20f4dd83592e862a72
content_type: application/pdf
creator: dernst
date_created: 2024-08-20T11:00:03Z
date_updated: 2024-08-20T11:00:03Z
file_id: '17451'
file_name: 2024_ReportPhysics_Brueckner.pdf
file_size: 4376898
relation: main_file
success: 1
file_date_updated: 2024-08-20T11:00:03Z
fulldoi: https://doi.org/10.1088/1361-6633/ad36d2
has_accepted_license: '1'
intvolume: ' 87'
isi: 1
issue: '5'
language:
- iso: eng
license: https://creativecommons.org/licenses/by/3.0/
month: '04'
oa: 1
oa_version: Published Version
pmid: 1
project:
- _id: 34e2a5b5-11ca-11ed-8bc3-b2265616ef0b
grant_number: ALTF 343-2022
name: A mechano-chemical theory for stem cell fate decisions in organoid development
publication: Reports on Progress in Physics
publication_identifier:
eissn:
- 1361-6633
issn:
- 0034-4885
publication_status: published
publisher: IOP Publishing
quality_controlled: '1'
researchdata_availability: not applicable
scopus_import: '1'
status: public
supplementarymaterial: no
title: 'Learning dynamical models of single and collective cell migration: a review'
tmp:
image: /images/cc_by.png
legal_code_url: https://creativecommons.org/licenses/by/3.0/legalcode
name: Creative Commons Attribution 3.0 Unported (CC BY 3.0)
short: CC BY (3.0)
type: journal_article
user_id: 317138e5-6ab7-11ef-aa6d-ffef3953e345
volume: 87
year: '2024'
...