SparseProp: Efficient sparse backpropagation for faster training of neural networks at the edge
M. Nikdan, T. Pegolotti, E.B. Iofinova, E. Kurtic, D.-A. Alistarh, in:, Proceedings of the 40th International Conference on Machine Learning, ML Research Press, 2023, pp. 26215–26227.

SparseGPT: Massive language models can be accurately pruned in one-shot
E. Frantar, D.-A. Alistarh, in:, Proceedings of the 40th International Conference on Machine Learning, ML Research Press, 2023, pp. 10323–10337.

Why extension-based proofs fail
D.-A. Alistarh, J. Aspnes, F. Ellen, R. Gelashvili, L. Zhu, SIAM Journal on Computing 52 (2023) 913–944.

Bias in pruned vision models: In-depth analysis and countermeasures
E.B. Iofinova, E.-A. Peste, D.-A. Alistarh, in:, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2023, pp. 24364–24373.

On biased compression for distributed learning
A. Beznosikov, S. Horvath, P. Richtarik, M. Safaryan, Journal of Machine Learning Research 24 (2023) 1–50.

Lincheck: A practical framework for testing concurrent data structures on JVM
N. Koval, A. Fedorov, M. Sokolova, D. Tsitelov, D.-A. Alistarh, in:, 35th International Conference on Computer Aided Verification , Springer Nature, 2023, pp. 156–169.

Lincheck: A practical framework for testing concurrent data structures on JVM
N. Koval, A. Fedorov, M. Sokolova, D. Tsitelov, D.-A. Alistarh, (2023).

Provably-efficient and internally-deterministic parallel Union-Find
A. Fedorov, D. Hashemi, G. Nadiradze, D.-A. Alistarh, in:, Proceedings of the 35th ACM Symposium on Parallelism in Algorithms and Architectures, Association for Computing Machinery, 2023, pp. 261–271.

Knowledge distillation performs partial variance reduction
M. Safaryan, A. Krumes, D.-A. Alistarh, in:, 36th Conference on Neural Information Processing Systems, 2023.

Fast graphical population protocols
D.-A. Alistarh, R. Gelashvili, J. Rybicki, in:, Q. Bramas, V. Gramoli, A. Milani (Eds.), 25th International Conference on Principles of Distributed Systems, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2022.

Beyond distributed subgraph detection: Induced subgraphs, multicolored problems and graph parameters
A. Nikabadi, J. Korhonen, in:, Q. Bramas, V. Gramoli, A. Milani (Eds.), 25th International Conference on Principles of Distributed Systems, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2022.

Mean-field analysis of piecewise linear solutions for wide ReLU networks
A. Shevchenko, V. Kungurtsev, M. Mondelli, Journal of Machine Learning Research 23 (2022) 1–55.

Brief announcement: Temporal locality in online algorithms
M. Pacut, M. Parham, J. Rybicki, S. Schmid, J. Suomela, A. Tereshchenko, in:, 36th International Symposium on Distributed Computing, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2022.

CGX: Adaptive system support for communication-efficient deep learning
I. Markov, H. Ramezanikebrya, D.-A. Alistarh, in:, Proceedings of the 23rd ACM/IFIP International Middleware Conference, Association for Computing Machinery, 2022, pp. 241–254.

Near-optimal leader election in population protocols on graphs
D.-A. Alistarh, J. Rybicki, S. Voitovych, in:, Proceedings of the Annual ACM Symposium on Principles of Distributed Computing, Association for Computing Machinery, 2022, pp. 246–256.

PathCAS: An efficient middle ground for concurrent search data structures
T.A. Brown, W. Sigouin, D.-A. Alistarh, in:, Proceedings of the 27th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, Association for Computing Machinery, 2022, pp. 385–399.

Multi-queues can be state-of-the-art priority schedulers
A. Postnikova, N. Koval, G. Nadiradze, D.-A. Alistarh, in:, Proceedings of the 27th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, Association for Computing Machinery, 2022, pp. 353–367.

Multi-queues can be state-of-the-art priority schedulers
A. Postnikova, N. Koval, G. Nadiradze, D.-A. Alistarh, (2022).

Local mending
A. Balliu, J. Hirvonen, D. Melnyk, D. Olivetti, J. Rybicki, J. Suomela, in:, M. Parter (Ed.), International Colloquium on Structural Information and Communication Complexity, Springer Nature, 2022, pp. 1–20.

How well do sparse ImageNet models transfer?
E.B. Iofinova, E.-A. Peste, M. Kurtz, D.-A. Alistarh, in:, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Institute of Electrical and Electronics Engineers, 2022, pp. 12256–12266.

Dynamic averaging load balancing on cycles
D.-A. Alistarh, G. Nadiradze, A. Sabour, Algorithmica 84 (2022) 1007–1029.

Sparsity in deep learning: Pruning and growth for efficient inference and training in neural networks
T. Hoefler, D.-A. Alistarh, T. Ben-Nun, N. Dryden, E.-A. Peste, Journal of Machine Learning Research 22 (2021) 1–124.

Brief announcement: Fast graphical population protocols
D.-A. Alistarh, R. Gelashvili, J. Rybicki, in:, 35th International Symposium on Distributed Computing, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2021.

Lower bounds for shared-memory leader election under bounded write contention
D.-A. Alistarh, R. Gelashvili, G. Nadiradze, in:, 35th International Symposium on Distributed Computing, Schloss Dagstuhl - Leibniz Zentrum für Informatik, 2021.

Brief announcement: Non-blocking dynamic unbounded graphs with worst-case amortized bounds
B. Chatterjee, S. Peri, M. Sa, in:, 35th International Symposium on Distributed Computing, Schloss Dagstuhl - Leibniz Zentrum für Informatik, 2021.

Brief announcement: Sinkless orientation is hard also in the supported LOCAL model
J. Korhonen, A. Paz, J. Rybicki, S. Schmid, J. Suomela, in:, 35th International Symposium on Distributed Computing, Schloss Dagstuhl - Leibniz Zentrum für Informatik, 2021.

A scalable concurrent algorithm for dynamic connectivity
A. Fedorov, N. Koval, D.-A. Alistarh, in:, Proceedings of the 33rd ACM Symposium on Parallelism in Algorithms and Architectures, Association for Computing Machinery, 2021, pp. 208–220.

Asynchronous optimization methods for efficient training of deep neural networks with guarantees
V. Kungurtsev, M. Egan, B. Chatterjee, D.-A. Alistarh, in:, 35th AAAI Conference on Artificial Intelligence, AAAI 2021, AAAI Press, 2021, pp. 8209–8216.

Distributed principal component analysis with limited communication
F. Alimisis, P. Davies, B. Vandereycken, D.-A. Alistarh, in:, Advances in Neural Information Processing Systems - 35th Conference on Neural Information Processing Systems, Neural Information Processing Systems Foundation, 2021, pp. 2823–2834.

M-FAC: Efficient matrix-free approximations of second-order information
E. Frantar, E. Kurtic, D.-A. Alistarh, in:, 35th Conference on Neural Information Processing Systems, Curran Associates, 2021, pp. 14873–14886.

Towards tight communication lower bounds for distributed optimisation
D.-A. Alistarh, J. Korhonen, in:, 35th Conference on Neural Information Processing Systems, Curran Associates, 2021, pp. 7254–7266.

New bounds for distributed mean estimation and variance reduction
P. Davies, V. Gurunanthan, N. Moshrefi, S. Ashkboos, D.-A. Alistarh, in:, 9th International Conference on Learning Representations, 2021.

Collecting coupons is faster with friends
D.-A. Alistarh, P. Davies, in:, Structural Information and Communication Complexity, Springer Nature, 2021, pp. 3–12.

Wait-free approximate agreement on graphs
D.-A. Alistarh, F. Ellen, J. Rybicki, in:, Structural Information and Communication Complexity, Springer Nature, 2021, pp. 87–105.

AC/DC: Alternating Compressed/DeCompressed training of deep neural networks
E.-A. Peste, E.B. Iofinova, A. Vladu, D.-A. Alistarh, in:, 35th Conference on Neural Information Processing Systems, Curran Associates, 2021, pp. 8557–8570.

Communication-efficient distributed optimization with quantized preconditioners
F. Alimisis, P. Davies, D.-A. Alistarh, in:, Proceedings of the 38th International Conference on Machine Learning, ML Research Press, 2021, pp. 196–206.

Breaking (global) barriers in parallel stochastic optimization with wait-avoiding group averaging
S. Li, T.B.-N. Tal Ben-Nun, G. Nadiradze, S.D. Girolamo, N. Dryden, D.-A. Alistarh, T. Hoefler, IEEE Transactions on Parallel and Distributed Systems 32 (2021).

Concurrent linearizable nearest neighbour search in LockFree-kD-tree
B. Chatterjee, I. Walulya, P. Tsigas, Theoretical Computer Science 886 (2021) 27–48.

Comparison dynamics in population protocols
D.-A. Alistarh, M. Töpfer, P. Uznaski, in:, Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing, Association for Computing Machinery, 2021, pp. 55–65.

Improved deterministic (+1) coloring in low-space MPC
A. Czumaj, P. Davies, M. Parter, in:, Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing, Association for Computing Machinery, 2021, pp. 469–479.

Component stability in low-space massively parallel computation
A. Czumaj, P. Davies, M. Parter, in:, Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing, Association for Computing Machinery, 2021, pp. 481–491.

Elastic consistency: A practical consistency model for distributed stochastic gradient descent
G. Nadiradze, I. Markov, B. Chatterjee, V. Kungurtsev, D.-A. Alistarh, in:, Proceedings of the AAAI Conference on Artificial Intelligence, 2021, pp. 9037–9045.

Keep the dirt: tainted TreeKEM, adaptively and actively secure continuous group key agreement
K. Klein, G. Pascual Perez, M. Walter, C. Kamath Hosdurg, M. Capretto, M. Cueto Noval, I. Markov, M.X. Yeo, J.F. Alwen, K.Z. Pietrzak, in:, 2021 IEEE Symposium on Security and Privacy , IEEE, 2021, pp. 268–284.

Input-dynamic distributed algorithms for communication networks
K.-T. Foerster, J. Korhonen, A. Paz, J. Rybicki, S. Schmid, in:, Abstract Proceedings of the 2021 ACM SIGMETRICS / International Conference on Measurement and Modeling of Computer Systems, Association for Computing Machinery, 2021, pp. 71–72.

Input-dynamic distributed algorithms for communication networks
K.-T. Foerster, J. Korhonen, A. Paz, J. Rybicki, S. Schmid, Proceedings of the ACM on Measurement and Analysis of Computing Systems 5 (2021) 1–33.

On achieving scalability through relaxation
G. Nadiradze, On Achieving Scalability through Relaxation, Institute of Science and Technology Austria, 2021.

Asynchronous decentralized SGD with quantized and local updates
G. Nadiradze, A. Sabour, P. Davies, S. Li, D.-A. Alistarh, in:, 35th Conference on Neural Information Processing Systems, Neural Information Processing Systems Foundation, 2021.

Graph sparsification for derandomizing massively parallel computation with low space
A. Czumaj, P. Davies, M. Parter, ACM Transactions on Algorithms 17 (2021).

Efficient load-balancing through distributed token dropping
S. Brandt, B. Keller, J. Rybicki, J. Suomela, J. Uitto, in:, Annual ACM Symposium on Parallelism in Algorithms and Architectures, 2021, pp. 129–139.

NUQSGD: Provably communication-efficient data-parallel SGD via nonuniform quantization
A. Ramezani-Kebrya, F. Faghri, I. Markov, V. Aksenov, D.-A. Alistarh, D.M. Roy, Journal of Machine Learning Research 22 (2021) 143.

Fast approximate shortest paths in the congested clique
K. Censor-Hillel, M. Dory, J. Korhonen, D. Leitersdorf, Distributed Computing 34 (2021) 463–487.

Simple, deterministic, constant-round coloring in congested clique and MPC
A. Czumaj, P. Davies, M. Parter, SIAM Journal on Computing 50 (2021) 1603–1626.

Exploiting spontaneous transmissions for broadcasting and leader election in radio networks
A. Czumaj, P. Davies, Journal of the ACM 68 (2021).

Momentum improves optimization on Riemannian manifolds
F. Alimisis, A. Orvieto, G. Becigneul, A. Lucchi, in:, Proceedings of the 24th International Conference on Artificial Intelligence and Statistics, ML Research Press, 2021, pp. 1351–1359.

Getting to the root of concurrent binary search tree performance
M. Arbel-Raviv, T.A. Brown, A. Morrison, in:, Proceedings of the 2018 USENIX Annual Technical Conference, USENIX Association, 2020, pp. 295–306.

In search of the fastest concurrent union-find algorithm
D.-A. Alistarh, A. Fedorov, N. Koval, in:, 23rd International Conference on Principles of Distributed Systems, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2020, p. 15:1-15:16.

Simple, deterministic, constant-round coloring in the congested clique
A. Czumaj, P. Davies, M. Parter, in:, Proceedings of the 2020 ACM Symposium on Principles of Distributed Computing, Association for Computing Machinery, 2020, pp. 309–318.

The splay-list: A distribution-adaptive concurrent skip-list
V. Aksenov, D.-A. Alistarh, A. Drozdova, A. Mohtashami, in:, 34th International Symposium on Distributed Computing, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2020, p. 3:1-3:18.

WoodFisher: Efficient second-order approximation for neural network compression
S.P. Singh, D.-A. Alistarh, in:, Advances in Neural Information Processing Systems, Curran Associates, 2020, pp. 18098–18109.

Scalable belief propagation via relaxed scheduling
V. Aksenov, D.-A. Alistarh, J. Korhonen, in:, Advances in Neural Information Processing Systems, Curran Associates, 2020, pp. 22361–22372.

Inducing and exploiting activation sparsity for fast neural network inference
M. Kurtz, J. Kopinsky, R. Gelashvili, A. Matveev, J. Carr, M. Goin, W. Leiserson, S. Moore, B. Nell, N. Shavit, D.-A. Alistarh, in:, 37th International Conference on Machine Learning, ICML 2020, 2020, pp. 5533–5543.

Compressive sensing using iterative hard thresholding with low precision data representation: Theory and applications
N.M. Gurel, K. Kara, A. Stojanov, T. Smith, T. Lemmin, D.-A. Alistarh, M. Puschel, C. Zhang, IEEE Transactions on Signal Processing 68 (2020) 4268–4282.

Taming unbalanced training workloads in deep learning with partial collective operations
S. Li, T.B.-N. Tal Ben-Nun, S.D. Girolamo, D.-A. Alistarh, T. Hoefler, in:, Proceedings of the 25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, Association for Computing Machinery, 2020, pp. 45–61.

Habitat fragmentation and species diversity in competitive communities
J. Rybicki, N. Abrego, O. Ovaskainen, Ecology Letters 23 (2020) 506–517.

On the sample complexity of adversarial multi-source PAC learning
N.H. Konstantinov, E. Frantar, D.-A. Alistarh, C. Lampert, in:, Proceedings of the 37th International Conference on Machine Learning, ML Research Press, 2020, pp. 5416–5425.

A persistent homology perspective to the link prediction problem
S. Bhatia, B. Chatterjee, D. Nathani, M. Kaul, in:, Complex Networks and Their Applications VIII, Springer Nature, 2020, pp. 27–39.

Graph sparsification for derandomizing massively parallel computation with low space
A. Czumaj, P. Davies, M. Parter, in:, Proceedings of the 32nd ACM Symposium on Parallelism in Algorithms and Architectures (SPAA 2020), Association for Computing Machinery, 2020, pp. 175–185.

Non-blocking interpolation search trees with doubly-logarithmic running time
T.A. Brown, A. Prokopec, D.-A. Alistarh, in:, Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, Association for Computing Machinery, 2020, pp. 276–291.

Memory tagging: Minimalist synchronization for scalable concurrent data structures
D.-A. Alistarh, T.A. Brown, N. Singhal, in:, Annual ACM Symposium on Parallelism in Algorithms and Architectures, Association for Computing Machinery, 2020, pp. 37–49.

Testing concurrency on the JVM with Lincheck
N. Koval, M. Sokolova, A. Fedorov, D.-A. Alistarh, D. Tsitelov, in:, Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPOPP, Association for Computing Machinery, 2020, pp. 423–424.

Brief Announcement: Why Extension-Based Proofs Fail
D.-A. Alistarh, J. Aspnes, F. Ellen, R. Gelashvili, L. Zhu, in:, Proceedings of the 39th Symposium on Principles of Distributed Computing, Association for Computing Machinery, 2020, pp. 54–56.

Brief announcement: Efficient load-balancing through distributed token dropping
S. Brandt, B. Keller, J. Rybicki, J. Suomela, J. Uitto, in:, 34th International Symposium on Distributed Computing, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2020.

Adaptive gradient quantization for data-parallel SGD
F. Faghri, I. Tabrizian, I. Markov, D.-A. Alistarh, D. Roy, A. Ramezani-Kebrya, in:, Advances in Neural Information Processing Systems, Neural Information Processing Systems Foundation, 2020.

Dynamic averaging load balancing on cycles
D.-A. Alistarh, G. Nadiradze, A. Sabour, in:, 47th International Colloquium on Automata, Languages, and Programming, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2020.

On grounded L-graphs and their relatives
V. Jelínek, M. Töpfer, Electronic Journal of Combinatorics 26 (2019).

Byzantine approximate agreement on graphs
T. Nowak, J. Rybicki, in:, 33rd International Symposium on Distributed Computing, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2019, p. 29:1--29:17.

A simple and practical concurrent non-blocking unbounded graph with linearizable reachability queries
B. Chatterjee, S. Peri, M. Sa, N. Singhal, in:, ACM International Conference Proceeding Series, ACM, 2019, pp. 168–177.

Lock-free channels for programming via communicating sequential processes
N. Koval, D.-A. Alistarh, R. Elizarov, Lock-Free Channels for Programming via Communicating Sequential Processes, ACM Press, 2019.

What can observational data reveal about metacommunity processes?
O. Ovaskainen, J. Rybicki, N. Abrego, Ecography 42 (2019) 1877–1886.

Self-stabilising Byzantine clock synchronisation is almost as easy as consensus
C. Lenzen, J. Rybicki, Journal of the ACM 66 (2019).

Gradient compression for communication-limited convex optimization
S. Khirirat, M. Johansson, D.-A. Alistarh, in:, 2018 IEEE Conference on Decision and Control, IEEE, 2019.

SparCML: High-performance sparse communication for machine learning
C. Renggli, S. Ashkboos, M. Aghagolzadeh, D.-A. Alistarh, T. Hoefler, in:, International Conference for High Performance Computing, Networking, Storage and Analysis, SC, ACM, 2019.

Recovering rearranged cancer chromosomes from karyotype graphs
S. Aganezov, I. Zban, V. Aksenov, N. Alexeev, M.C. Schatz, BMC Bioinformatics 20 (2019).

Scalable FIFO channels for programming via communicating sequential processes
N. Koval, D.-A. Alistarh, R. Elizarov, in:, 25th Anniversary of Euro-Par, Springer Nature, 2019, pp. 317–333.

Distributed learning over unreliable networks
C. Yu, H. Tang, C. Renggli, S. Kassing, A. Singla, D.-A. Alistarh, C. Zhang, J. Liu, in:, 36th International Conference on Machine Learning, ICML 2019, IMLS, 2019, pp. 12481–12512.

Efficiency guarantees for parallel incremental algorithms under relaxed schedulers
D.-A. Alistarh, G. Nadiradze, N. Koval, in:, 31st ACM Symposium on Parallelism in Algorithms and Architectures, ACM Press, 2019, pp. 145–154.

Powerset convolutional neural networks
C. Wendler, D.-A. Alistarh, M. Püschel, in:, Neural Information Processing Systems Foundation, 2019, pp. 927–938.

Does preprocessing help under congestion?
K.-T. Foerster, J. Korhonen, J. Rybicki, S. Schmid, in:, Proceedings of the 2019 ACM Symposium on Principles of Distributed Computing, ACM, 2019, pp. 259–261.

Why extension-based proofs fail
D.-A. Alistarh, J. Aspnes, F. Ellen, R. Gelashvili, L. Zhu, in:, Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing, ACM Press, 2019, pp. 986–996.

Fast approximate shortest paths in the congested clique
K. Censor-Hillel, M. Dory, J. Korhonen, D. Leitersdorf, in:, Proceedings of the 2019 ACM Symposium on Principles of Distributed Computin, ACM, 2019, pp. 74–83.

Communication-efficient randomized consensus
D.-A. Alistarh, J. Aspnes, V. King, J. Saia, Distributed Computing 31 (2018) 489–501.

Synchronous multi-GPU training for deep learning with low-precision communications: An empirical study
D. Grubic, L. Tam, D.-A. Alistarh, C. Zhang, in:, Proceedings of the 21st International Conference on Extending Database Technology, OpenProceedings, 2018, pp. 145–156.

ThreadScan: Automatic and scalable memory reclamation
D.-A. Alistarh, W. Leiserson, A. Matveev, N. Shavit, ACM Transactions on Parallel Computing 4 (2018).

Model compression via distillation and quantization
A. Polino, R. Pascanu, D.-A. Alistarh, in:, 6th International Conference on Learning Representations, 2018.

Harnessing epoch-based reclamation for efficient range queries
M. Arbel Raviv, T.A. Brown, in:, ACM, 2018, pp. 14–27.

Model of bacterial toxin-dependent pathogenesis explains infective dose
J. Rybicki, E. Kisdi, J. Anttila, PNAS 115 (2018) 10690–10695.

Near-optimal self-stabilising counting and firing squads
C. Lenzen, J. Rybicki, Distributed Computing (2018).

Snapshot based synchronization: A fast replacement for Hand-over-Hand locking
E. Gilad, T.A. Brown, M. Oskin, Y. Etsion, in:, Springer, 2018, pp. 465–479.

The convergence of stochastic gradient descent in asynchronous shared memory
D.-A. Alistarh, C. De Sa, N.H. Konstantinov, in:, Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18, ACM Press, 2018, pp. 169–178.

A brief tutorial on distributed and concurrent machine learning
D.-A. Alistarh, in:, Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18, ACM Press, 2018, pp. 487–488.