Please note that ISTA Research Explorer no longer supports Internet Explorer versions 8 or 9 (or earlier).
We recommend upgrading to the latest Internet Explorer, Google Chrome, or Firefox.
138 Publications
2024 |Published| Conference Paper | IST-REx-ID: 15011 |
Kurtic E, Hoefler T, Alistarh D-A. How to prune your language model: Recovering accuracy on the “Sparsity May Cry” benchmark. In: Proceedings of Machine Learning Research. Vol 234. ML Research Press; 2024:542-553.
[Preprint]
View
| Download Preprint (ext.)
| arXiv
2024 |Published| Conference Paper | IST-REx-ID: 17093 |
Zakerinia H, Talaei S, Nadiradze G, Alistarh D-A. Communication-efficient federated learning with data and client heterogeneity. In: Proceedings of the 27th International Conference on Artificial Intelligence and Statistics. Vol 238. ML Research Press; 2024:3448-3456.
[Preprint]
View
| Download Preprint (ext.)
| arXiv
2024 |Published| Conference Paper | IST-REx-ID: 17329 |
Alistarh D-A, Chatterjee K, Karrabi M, Lazarsfeld JM. Game dynamics and equilibrium computation in the population protocol model. In: Proceedings of the 43rd Annual ACM Symposium on Principles of Distributed Computing. Association for Computing Machinery; 2024:40-49. doi:10.1145/3662158.3662768
[Published Version]
View
| Files available
| DOI
2024 |Published| Conference Paper | IST-REx-ID: 17332 |
Kokorin I, Yudov V, Aksenov V, Alistarh D-A. Wait-free trees with asymptotically-efficient range queries. In: 2024 IEEE International Parallel and Distributed Processing Symposium. IEEE; 2024:169-179. doi:10.1109/IPDPS57955.2024.00023
[Preprint]
View
| DOI
| Download Preprint (ext.)
| arXiv
2024 |Published| Conference Paper | IST-REx-ID: 17469 |
Kögler K, Shevchenko A, Hassani H, Mondelli M. Compression of structured data with autoencoders: Provable benefit of nonlinearities and depth. In: Proceedings of the 41st International Conference on Machine Learning. Vol 235. ML Research Press; 2024:24964-25015.
[Published Version]
View
| Files available
| Download Published Version (ext.)
| arXiv
2024 |Published| Thesis | IST-REx-ID: 17465
Shevchenko A. High-dimensional limits in artificial neural networks. 2024. doi:10.15479/at:ista:17465
[Published Version]
View
| Files available
| DOI
2024 |Published| Thesis | IST-REx-ID: 17490 |
Markov I. Communication-efficient distributed training of deep neural networks: An algorithms and systems perspective. 2024. doi:10.15479/at:ista:17490
[Published Version]
View
| Files available
| DOI
2024 |Published| Conference Paper | IST-REx-ID: 17456 |
Markov I, Alimohammadi K, Frantar E, Alistarh D-A. L-GreCo: Layerwise-adaptive gradient compression for efficient data-parallel deep learning. In: Gibbons P, Pekhimenko G, De Sa C, eds. Proceedings of Machine Learning and Systems . Vol 6. Association for Computing Machinery; 2024.
[Published Version]
View
| Files available
| Download Published Version (ext.)
| arXiv
2024 |Published| Conference Paper | IST-REx-ID: 18070
Chatterjee B, Kungurtsev V, Alistarh D-A. Federated SGD with local asynchrony. In: Proceedings of the 44th International Conference on Distributed Computing Systems. IEEE; 2024:857-868. doi:10.1109/ICDCS60910.2024.00084
View
| DOI
2024 |Published| Thesis | IST-REx-ID: 17485 |
Frantar E. Compressing large neural networks : Algorithms, systems and scaling laws. 2024. doi:10.15479/at:ista:17485
[Published Version]
View
| Files available
| DOI
2024 |Published| Conference Paper | IST-REx-ID: 18061 |
Frantar E, Alistarh D-A. QMoE: Sub-1-bit compression of trillion parameter models. In: Gibbons P, Pekhimenko G, De Sa C, eds. Proceedings of Machine Learning and Systems. Vol 6. ; 2024.
[Published Version]
View
| Files available
| Download Published Version (ext.)
2024 |Published| Conference Paper | IST-REx-ID: 18062 |
Frantar E, Ruiz CR, Houlsby N, Alistarh D-A, Evci U. Scaling laws for sparsely-connected foundation models. In: The Twelfth International Conference on Learning Representations. ; 2024.
[Published Version]
View
| Files available
| Download Published Version (ext.)
| arXiv
2023 |Published| Conference Paper | IST-REx-ID: 12735 |
Koval N, Alistarh D-A, Elizarov R. Fast and scalable channels in Kotlin Coroutines. In: Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. Association for Computing Machinery; 2023:107-118. doi:10.1145/3572848.3577481
[Preprint]
View
| DOI
| Download Preprint (ext.)
| arXiv
2023 |Published| Conference Poster | IST-REx-ID: 12736 |
Aksenov V, Brown TA, Fedorov A, Kokorin I. Unexpected Scaling in Path Copying Trees. Association for Computing Machinery; 2023:438-440. doi:10.1145/3572848.3577512
[Published Version]
View
| DOI
| Download Published Version (ext.)
2023 |Published| Journal Article | IST-REx-ID: 13179 |
Koval N, Khalanskiy D, Alistarh D-A. CQS: A formally-verified framework for fair and abortable synchronization. Proceedings of the ACM on Programming Languages. 2023;7. doi:10.1145/3591230
[Published Version]
View
| Files available
| DOI
2023 |Published| Journal Article | IST-REx-ID: 12566 |
Alistarh D-A, Ellen F, Rybicki J. Wait-free approximate agreement on graphs. Theoretical Computer Science. 2023;948(2). doi:10.1016/j.tcs.2023.113733
[Published Version]
View
| Files available
| DOI
| WoS
2023 |Published| Journal Article | IST-REx-ID: 12330 |
Aksenov V, Alistarh D-A, Drozdova A, Mohtashami A. The splay-list: A distribution-adaptive concurrent skip-list. Distributed Computing. 2023;36:395-418. doi:10.1007/s00446-022-00441-x
[Preprint]
View
| DOI
| Download Preprint (ext.)
| WoS
| arXiv
2023 |Published| Conference Paper | IST-REx-ID: 14460 |
Nikdan M, Pegolotti T, Iofinova EB, Kurtic E, Alistarh D-A. SparseProp: Efficient sparse backpropagation for faster training of neural networks at the edge. In: Proceedings of the 40th International Conference on Machine Learning. Vol 202. ML Research Press; 2023:26215-26227.
[Preprint]
View
| Download Preprint (ext.)
| arXiv
2023 |Published| Journal Article | IST-REx-ID: 14364 |
Alistarh D-A, Aspnes J, Ellen F, Gelashvili R, Zhu L. Why extension-based proofs fail. SIAM Journal on Computing. 2023;52(4):913-944. doi:10.1137/20M1375851
[Preprint]
View
| Files available
| DOI
| Download Preprint (ext.)
| WoS
| arXiv
2023 |Published| Conference Paper | IST-REx-ID: 14771 |
Iofinova EB, Peste E-A, Alistarh D-A. Bias in pruned vision models: In-depth analysis and countermeasures. In: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE; 2023:24364-24373. doi:10.1109/cvpr52729.2023.02334
[Preprint]
View
| Files available
| DOI
| Download Preprint (ext.)
| WoS
| arXiv
2023 |Published| Journal Article | IST-REx-ID: 14815 |
Beznosikov A, Horvath S, Richtarik P, Safaryan M. On biased compression for distributed learning. Journal of Machine Learning Research. 2023;24:1-50.
[Published Version]
View
| Files available
| WoS
| arXiv
2023 |Published| Conference Paper | IST-REx-ID: 14260 |
Koval N, Fedorov A, Sokolova M, Tsitelov D, Alistarh D-A. Lincheck: A practical framework for testing concurrent data structures on JVM. In: 35th International Conference on Computer Aided Verification . Vol 13964. Springer Nature; 2023:156-169. doi:10.1007/978-3-031-37706-8_8
[Published Version]
View
| Files available
| DOI
2023 | Research Data Reference | IST-REx-ID: 14995 |
Koval N, Fedorov A, Sokolova M, Tsitelov D, Alistarh D-A. Lincheck: A practical framework for testing concurrent data structures on JVM. 2023. doi:10.5281/ZENODO.7877757
[Published Version]
View
| Files available
| DOI
| Download Published Version (ext.)
2023 |Published| Conference Paper | IST-REx-ID: 13262 |
Fedorov A, Hashemi D, Nadiradze G, Alistarh D-A. Provably-efficient and internally-deterministic parallel Union-Find. In: Proceedings of the 35th ACM Symposium on Parallelism in Algorithms and Architectures. Association for Computing Machinery; 2023:261-271. doi:10.1145/3558481.3591082
[Published Version]
View
| Files available
| DOI
| arXiv
2023 |Published| Conference Paper | IST-REx-ID: 15363 |
Safaryan M, Krumes A, Alistarh D-A. Knowledge distillation performs partial variance reduction. In: 36th Conference on Neural Information Processing Systems. Vol 36. ; 2023.
[Published Version]
View
| Files available
| arXiv
2023 |Published| Thesis | IST-REx-ID: 13074 |
Peste E-A. Efficiency and generalization of sparse neural networks. 2023. doi:10.15479/at:ista:13074
[Published Version]
View
| Files available
| DOI
2023 |Published| Conference Paper | IST-REx-ID: 13053 |
Krumes A, Vladu A, Kurtic E, Lampert C, Alistarh D-A. CrAM: A Compression-Aware Minimizer. In: 11th International Conference on Learning Representations . OpenReview; 2023.
[Published Version]
View
| Files available
| Download Published Version (ext.)
| arXiv
2023 |Published| Conference Paper | IST-REx-ID: 14459 |
Shevchenko A, Kögler K, Hassani H, Mondelli M. Fundamental limits of two-layer autoencoders, and achieving them with gradient methods. In: Proceedings of the 40th International Conference on Machine Learning. Vol 202. ML Research Press; 2023:31151-31209.
[Preprint]
View
| Files available
| Download Preprint (ext.)
| arXiv
2023 |Published| Conference Paper | IST-REx-ID: 14461 |
Markov I, Vladu A, Guo Q, Alistarh D-A. Quantized distributed training of large models with convergence guarantees. In: Proceedings of the 40th International Conference on Machine Learning. Vol 202. ML Research Press; 2023:24020-24044.
[Preprint]
View
| Files available
| Download Preprint (ext.)
| arXiv
2023 |Published| Conference Paper | IST-REx-ID: 17378 |
Frantar E, Ashkboos S, Hoefler T, Alistarh D-A. OPTQ: Accurate post-training quantization for generative pre-trained transformers. In: 11th International Conference on Learning Representations . International Conference on Learning Representations; 2023.
[Published Version]
View
| Files available
2023 |Published| Conference Paper | IST-REx-ID: 14458 |
Frantar E, Alistarh D-A. SparseGPT: Massive language models can be accurately pruned in one-shot. In: Proceedings of the 40th International Conference on Machine Learning. Vol 202. ML Research Press; 2023:10323-10337.
[Preprint]
View
| Files available
| Download Preprint (ext.)
| arXiv
2022 |Published| Conference Paper | IST-REx-ID: 11184 |
Alistarh D-A, Gelashvili R, Rybicki J. Fast graphical population protocols. In: Bramas Q, Gramoli V, Milani A, eds. 25th International Conference on Principles of Distributed Systems. Vol 217. Schloss Dagstuhl - Leibniz-Zentrum für Informatik; 2022. doi:10.4230/LIPIcs.OPODIS.2021.14
[Published Version]
View
| Files available
| DOI
| arXiv
2022 |Published| Conference Paper | IST-REx-ID: 11183 |
Nikabadi A, Korhonen J. Beyond distributed subgraph detection: Induced subgraphs, multicolored problems and graph parameters. In: Bramas Q, Gramoli V, Milani A, eds. 25th International Conference on Principles of Distributed Systems. Vol 217. Schloss Dagstuhl - Leibniz-Zentrum für Informatik; 2022. doi:10.4230/LIPIcs.OPODIS.2021.15
[Published Version]
View
| Files available
| DOI
2022 |Published| Conference Paper | IST-REx-ID: 12182 |
Pacut M, Parham M, Rybicki J, Schmid S, Suomela J, Tereshchenko A. Brief announcement: Temporal locality in online algorithms. In: 36th International Symposium on Distributed Computing. Vol 246. Schloss Dagstuhl - Leibniz-Zentrum für Informatik; 2022. doi:10.4230/LIPIcs.DISC.2022.52
[Published Version]
View
| Files available
| DOI
2022 |Published| Conference Paper | IST-REx-ID: 11844 |
Alistarh D-A, Rybicki J, Voitovych S. Near-optimal leader election in population protocols on graphs. In: Proceedings of the Annual ACM Symposium on Principles of Distributed Computing. Association for Computing Machinery; 2022:246-256. doi:10.1145/3519270.3538435
[Published Version]
View
| Files available
| DOI
| arXiv
2022 |Published| Conference Paper | IST-REx-ID: 11181 |
Brown TA, Sigouin W, Alistarh D-A. PathCAS: An efficient middle ground for concurrent search data structures. In: Proceedings of the 27th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. Association for Computing Machinery; 2022:385-399. doi:10.1145/3503221.3508410
[Published Version]
View
| Files available
| DOI
| WoS
2022 |Published| Conference Paper | IST-REx-ID: 11180 |
Postnikova A, Koval N, Nadiradze G, Alistarh D-A. Multi-queues can be state-of-the-art priority schedulers. In: Proceedings of the 27th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. Association for Computing Machinery; 2022:353-367. doi:10.1145/3503221.3508432
[Preprint]
View
| Files available
| DOI
| Download Preprint (ext.)
| WoS
| arXiv
2022 | Research Data Reference | IST-REx-ID: 13076 |
Postnikova A, Koval N, Nadiradze G, Alistarh D-A. Multi-queues can be state-of-the-art priority schedulers. 2022. doi:10.5281/ZENODO.5733408
[Published Version]
View
| Files available
| DOI
| Download Published Version (ext.)
2022 |Published| Conference Paper | IST-REx-ID: 11707 |
Balliu A, Hirvonen J, Melnyk D, Olivetti D, Rybicki J, Suomela J. Local mending. In: Parter M, ed. International Colloquium on Structural Information and Communication Complexity. Vol 13298. LNCS. Springer Nature; 2022:1-20. doi:10.1007/978-3-031-09993-9_1
[Preprint]
View
| DOI
| Download Preprint (ext.)
| WoS
| arXiv
2022 |Published| Conference Paper | IST-REx-ID: 12299 |
Iofinova EB, Peste E-A, Kurtz M, Alistarh D-A. How well do sparse ImageNet models transfer? In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Institute of Electrical and Electronics Engineers; 2022:12256-12266. doi:10.1109/cvpr52688.2022.01195
[Preprint]
View
| Files available
| DOI
| Download Preprint (ext.)
| WoS
| arXiv
2022 |Published| Journal Article | IST-REx-ID: 8286 |
Alistarh D-A, Nadiradze G, Sabour A. Dynamic averaging load balancing on cycles. Algorithmica. 2022;84(4):1007-1029. doi:10.1007/s00453-021-00905-9
[Published Version]
View
| Files available
| DOI
| WoS
| arXiv
2022 |Published| Conference Paper | IST-REx-ID: 17088 |
Kurtic E, Campos D, Nguyen T, et al. The optimal BERT surgeon: Scalable and accurate second-order pruning for large language models. In: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics; 2022:4163-4181. doi:10.18653/v1/2022.emnlp-main.279
[Published Version]
View
| Files available
| DOI
| arXiv
2022 |Published| Conference Paper | IST-REx-ID: 17059 |
Frantar E, Alistarh D-A. SPDY: Accurate pruning with speedup guarantees. In: 39th International Conference on Machine Learning. Vol 162. ML Research Press; 2022:6726-6743.
[Published Version]
View
| Files available
| WoS
2022 |Published| Journal Article | IST-REx-ID: 11420 |
Shevchenko A, Kungurtsev V, Mondelli M. Mean-field analysis of piecewise linear solutions for wide ReLU networks. Journal of Machine Learning Research. 2022;23(130):1-55.
[Published Version]
View
| Files available
| arXiv
2022 |Published| Conference Paper | IST-REx-ID: 12780 |
Markov I, Ramezanikebrya H, Alistarh D-A. CGX: Adaptive system support for communication-efficient deep learning. In: Proceedings of the 23rd ACM/IFIP International Middleware Conference. Association for Computing Machinery; 2022:241-254. doi:10.1145/3528535.3565248
[Published Version]
View
| Files available
| DOI
| arXiv
2022 |Published| Conference Paper | IST-REx-ID: 17087 |
Frantar E, Singh SP, Alistarh D-A. Optimal brain compression: A framework for accurate post-training quantization and pruning. In: 36th Conference on Neural Information Processing Systems. Vol 35. ML Research Press; 2022.
[Submitted Version]
View
| Files available
| arXiv
2021 |Published| Journal Article | IST-REx-ID: 10180 |
Hoefler T, Alistarh D-A, Ben-Nun T, Dryden N, Peste E-A. Sparsity in deep learning: Pruning and growth for efficient inference and training in neural networks. Journal of Machine Learning Research. 2021;22(241):1-124.
[Published Version]
View
| Files available
| Download Published Version (ext.)
| arXiv
2021 |Published| Conference Paper | IST-REx-ID: 10218 |
Alistarh D-A, Gelashvili R, Rybicki J. Brief announcement: Fast graphical population protocols. In: 35th International Symposium on Distributed Computing. Vol 209. Schloss Dagstuhl - Leibniz-Zentrum für Informatik; 2021. doi:10.4230/LIPIcs.DISC.2021.43
[Published Version]
View
| Files available
| DOI
| arXiv
2021 |Published| Conference Paper | IST-REx-ID: 10217 |
Alistarh D-A, Gelashvili R, Nadiradze G. Lower bounds for shared-memory leader election under bounded write contention. In: 35th International Symposium on Distributed Computing. Vol 209. Schloss Dagstuhl - Leibniz Zentrum für Informatik; 2021. doi:10.4230/LIPIcs.DISC.2021.4
[Published Version]
View
| Files available
| DOI
2021 |Published| Conference Paper | IST-REx-ID: 10216 |
Chatterjee B, Peri S, Sa M. Brief announcement: Non-blocking dynamic unbounded graphs with worst-case amortized bounds. In: 35th International Symposium on Distributed Computing. Vol 209. Schloss Dagstuhl - Leibniz Zentrum für Informatik; 2021. doi:10.4230/LIPIcs.DISC.2021.52
[Published Version]
View
| Files available
| DOI
| arXiv