Hybrid decentralized optimization: Leveraging both first- and zeroth-order optimizers for faster convergence
S. Talaei, M. Ansaripour, G. Nadiradze, D.-A. Alistarh, Proceedings of The39th AAAI Conference on Artificial Intelligence 39 (2025) 20778–20786.

MARLIN: Mixed-precision auto-regressive parallel inference on Large Language Models
E. Frantar, R.L. Castro, J. Chen, T. Hoefler, D.-A. Alistarh, in:, Proceedings of the 30th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming, Association for Computing Machinery, 2025, pp. 239–251.

Near-optimal leader election in population protocols on graphs
D.-A. Alistarh, J. Rybicki, S. Voitovych, Distributed Computing (2025).

The journey matters: Average parameter count over pre-training unifies sparse and dense scaling laws
T. Jin, A.I. Humayun, U. Evci, S. Subramanian, A. Yazdanbakhsh, D.-A. Alistarh, G.K. Dziugaite, in:, 13th International Conference on Learning Representations, ICLR, 2025, pp. 85165–85181.

Wasserstein distances, neuronal entanglement, and sparsity
S. Sawmya, L. Kong, I. Markov, D.-A. Alistarh, N. Shavit, in:, 13th International Conference on Learning Representations, ICLR, 2025, pp. 26244–26274.

Scalable mechanistic neural networks
J. Chen, D. Yao, A.A. Pervez, D.-A. Alistarh, F. Locatello, in:, 13th International Conference on Learning Representations, ICLR, 2025, pp. 63716–63737.

LDAdam: Adaptive optimization from low-dimensional gradient statistics
T. Robert, M. Safaryan, I.-V. Modoranu, D.-A. Alistarh, in:, 13th International Conference on Learning Representations, ICLR, 2025, pp. 101877–101913.

Communication-efficient federated learning with data and client heterogeneity
H. Zakerinia, S. Talaei, G. Nadiradze, D.-A. Alistarh, in:, Proceedings of the 27th International Conference on Artificial Intelligence and Statistics, ML Research Press, 2024, pp. 3448–3456.

Wait-free trees with asymptotically-efficient range queries
I. Kokorin, V. Yudov, V. Aksenov, D.-A. Alistarh, in:, 2024 IEEE International Parallel and Distributed Processing Symposium, IEEE, 2024, pp. 169–179.

How to prune your language model: Recovering accuracy on the "Sparsity May Cry" benchmark
E. Kurtic, T. Hoefler, D.-A. Alistarh, in:, Proceedings of Machine Learning Research, ML Research Press, 2024, pp. 542–553.

Federated SGD with local asynchrony
B. Chatterjee, V. Kungurtsev, D.-A. Alistarh, in:, Proceedings of the 44th International Conference on Distributed Computing Systems, IEEE, 2024, pp. 857–868.

Extreme compression of large language models via additive quantization
V. Egiazarian, A. Panferov, D. Kuznedelev, E. Frantar, A. Babenko, D.-A. Alistarh, in:, Proceedings of the 41st International Conference on Machine Learning, ML Research Press, 2024, pp. 12284–12303.

RoSA: Accurate parameter-efficient fine-tuning via robust adaptation
M. Nikdan, S. Tabesh, E. Crncevic, D.-A. Alistarh, in:, Proceedings of the 41st International Conference on Machine Learning, ML Research Press, 2024, pp. 38187–38206.

Error feedback can accurately compress preconditioners
I.-V. Modoranu, A. Kalinov, E. Kurtic, E. Frantar, D.-A. Alistarh, in:, 41st International Conference on Machine Learning, ML Research Press, 2024, pp. 35910–35933.

SpQR: A sparse-quantized representation for near-lossless LLM weight compression
T. Dettmers, R.A. Svirschevski, V. Egiazarian, D. Kuznedelev, E. Frantar, S. Ashkboos, A. Borzunov, T. Hoefler, D.-A. Alistarh, in:, 12th International Conference on Learning Representations, OpenReview, 2024.

QMoE: Sub-1-bit compression of trillion parameter models
E. Frantar, D.-A. Alistarh, in:, P. Gibbons, G. Pekhimenko, C. De Sa (Eds.),  Proceedings of Machine Learning and Systems, 2024.

Scaling laws for sparsely-connected foundation models
E. Frantar, C.R. Ruiz, N. Houlsby, D.-A. Alistarh, U. Evci, in:, The Twelfth International Conference on Learning Representations, 2024.

Game dynamics and equilibrium computation in the population protocol model
D.-A. Alistarh, K. Chatterjee, M. Karrabi, J.M. Lazarsfeld, in:, Proceedings of the 43rd Annual ACM Symposium on Principles of Distributed Computing, Association for Computing Machinery, 2024, pp. 40–49.

AsGrad: A sharp unified analysis of asynchronous-SGD algorithms
R. Islamov, M. Safaryan, D.-A. Alistarh, in:, Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, ML Research Press, 2024, pp. 649–657.

SPADE: Sparsity-guided debugging for deep neural networks
A.S. Moakhar, E.B. Iofinova, E. Frantar, D.-A. Alistarh, in:, Proceedings of the 41st International Conference on Machine Learning, ML Research Press, 2024, pp. 45955–45987.