[{"language":[{"iso":"eng"}],"day":"01","publication":"9th International Conference on Learning Representations","oa":1,"corr_author":"1","external_id":{"arxiv":["2002.09268"]},"ec_funded":1,"month":"05","department":[{"_id":"DaAl"}],"conference":{"name":"ICLR: International Conference on Learning Representations","start_date":"2021-05-03","end_date":"2021-05-07","location":"Virtual"},"date_published":"2021-05-01T00:00:00Z","article_processing_charge":"No","citation":{"apa":"Davies, P., Gurunanthan, V., Moshrefi, N., Ashkboos, S., &#38; Alistarh, D.-A. (2021). New bounds for distributed mean estimation and variance reduction. In <i>9th International Conference on Learning Representations</i>. Virtual.","chicago":"Davies, Peter, Vijaykrishna Gurunanthan, Niusha  Moshrefi, Saleh Ashkboos, and Dan-Adrian Alistarh. “New Bounds for Distributed Mean Estimation and Variance Reduction.” In <i>9th International Conference on Learning Representations</i>, 2021.","ista":"Davies P, Gurunanthan V, Moshrefi N, Ashkboos S, Alistarh D-A. 2021. New bounds for distributed mean estimation and variance reduction. 9th International Conference on Learning Representations. ICLR: International Conference on Learning Representations.","short":"P. Davies, V. Gurunanthan, N. Moshrefi, S. Ashkboos, D.-A. Alistarh, in:, 9th International Conference on Learning Representations, 2021.","ieee":"P. Davies, V. Gurunanthan, N. Moshrefi, S. Ashkboos, and D.-A. Alistarh, “New bounds for distributed mean estimation and variance reduction,” in <i>9th International Conference on Learning Representations</i>, Virtual, 2021.","ama":"Davies P, Gurunanthan V, Moshrefi N, Ashkboos S, Alistarh D-A. New bounds for distributed mean estimation and variance reduction. In: <i>9th International Conference on Learning Representations</i>. ; 2021.","mla":"Davies, Peter, et al. “New Bounds for Distributed Mean Estimation and Variance Reduction.” <i>9th International Conference on Learning Representations</i>, 2021."},"project":[{"grant_number":"754411","call_identifier":"H2020","name":"ISTplus - Postdoctoral Fellowships","_id":"260C2330-B435-11E9-9278-68D0E5697425"}],"date_updated":"2026-06-18T19:53:18Z","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","main_file_link":[{"open_access":"1","url":"https://openreview.net/pdf?id=t86MwoUCCNe"}],"arxiv":1,"title":"New bounds for distributed mean estimation and variance reduction","_id":"9543","ddc":["000"],"author":[{"orcid":"0000-0002-5646-9524","full_name":"Davies, Peter","id":"11396234-BB50-11E9-B24C-90FCE5697425","last_name":"Davies","first_name":"Peter"},{"full_name":"Gurunanthan, Vijaykrishna","last_name":"Gurunanthan","first_name":"Vijaykrishna"},{"first_name":"Niusha ","last_name":"Moshrefi","id":"4db776ff-ce15-11eb-96e3-bc2b90b01c16","full_name":"Moshrefi, Niusha "},{"first_name":"Saleh","last_name":"Ashkboos","id":"0D0A9058-257B-11EA-A937-9341C3D8BC8A","full_name":"Ashkboos, Saleh"},{"orcid":"0000-0003-3650-940X","full_name":"Alistarh, Dan-Adrian","first_name":"Dan-Adrian","last_name":"Alistarh","id":"4A899BFC-F248-11E8-B48F-1D18A9856A87"}],"date_created":"2021-06-10T19:46:08Z","status":"public","publication_status":"published","type":"conference","oa_version":"Published Version","quality_controlled":"1","year":"2021","abstract":[{"text":"We consider the problem ofdistributed mean estimation (DME), in which n machines are each given a local d-dimensional vector xv∈Rd, and must cooperate to estimate the mean of their inputs μ=1n∑nv=1xv, while minimizing total communication cost. DME is a fundamental construct in distributed machine learning, and there has been considerable work on variants of this problem, especially in the context of distributed variance reduction for stochastic gradients in parallel SGD. Previous work typically assumes an upper bound on the norm of the input vectors, and achieves an error bound in terms of this norm. However, in many real applications, the input vectors are concentrated around the correct output μ, but μ itself has large norm. In such cases, previous output error bounds perform poorly. In this paper, we show that output error bounds need not depend on input norm. We provide a method of quantization which allows distributed mean estimation to be performed with solution quality dependent only on the distance between inputs, not on input norm, and show an analogous result for distributed variance reduction. The technique is based on a new connection with lattice theory. We also provide lower bounds showing that the communication to error trade-off of our algorithms is asymptotically optimal. As the lattices achieving optimal bounds under l2-norm can be computationally impractical, we also present an extension which leverages easy-to-use cubic lattices, and is loose only up to a logarithmic factor ind. We show experimentally that our method yields practical improvements for common applications, relative to prior approaches.","lang":"eng"}]},{"file":[{"file_id":"9595","file_size":11237154,"content_type":"application/pdf","date_updated":"2021-06-23T07:09:41Z","file_name":"2021_JournalOfMachineLearningResearch_Ramezani-Kebrya.pdf","date_created":"2021-06-23T07:09:41Z","checksum":"6428aa8bcb67768b6949c99b55d5281d","success":1,"relation":"main_file","creator":"asandaue","access_level":"open_access"}],"volume":22,"publisher":"Journal of Machine Learning Research","month":"04","department":[{"_id":"DaAl"}],"external_id":{"arxiv":["1908.06077"]},"corr_author":"1","oa":1,"day":"01","main_file_link":[{"url":"https://www.jmlr.org/papers/v22/20-255.html","open_access":"1"}],"arxiv":1,"article_processing_charge":"No","has_accepted_license":"1","date_published":"2021-04-01T00:00:00Z","file_date_updated":"2021-06-23T07:09:41Z","intvolume":"        22","publication_status":"published","year":"2021","quality_controlled":"1","ddc":["000"],"tmp":{"image":"/images/cc_by.png","legal_code_url":"https://creativecommons.org/licenses/by/4.0/legalcode","short":"CC BY (4.0)","name":"Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)"},"language":[{"iso":"eng"}],"publication_identifier":{"eissn":["1533-7928"],"issn":["1532-4435"]},"scopus_import":"1","issue":"114","publication":"Journal of Machine Learning Research","date_updated":"2025-07-10T12:01:54Z","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","citation":{"apa":"Ramezani-Kebrya, A., Faghri, F., Markov, I., Aksenov, V., Alistarh, D.-A., &#38; Roy, D. M. (2021). NUQSGD: Provably communication-efficient data-parallel SGD via nonuniform quantization. <i>Journal of Machine Learning Research</i>. Journal of Machine Learning Research.","chicago":"Ramezani-Kebrya, Ali, Fartash Faghri, Ilya Markov, Vitalii Aksenov, Dan-Adrian Alistarh, and Daniel M. Roy. “NUQSGD: Provably Communication-Efficient Data-Parallel SGD via Nonuniform Quantization.” <i>Journal of Machine Learning Research</i>. Journal of Machine Learning Research, 2021.","short":"A. Ramezani-Kebrya, F. Faghri, I. Markov, V. Aksenov, D.-A. Alistarh, D.M. Roy, Journal of Machine Learning Research 22 (2021) 1−43.","ieee":"A. Ramezani-Kebrya, F. Faghri, I. Markov, V. Aksenov, D.-A. Alistarh, and D. M. Roy, “NUQSGD: Provably communication-efficient data-parallel SGD via nonuniform quantization,” <i>Journal of Machine Learning Research</i>, vol. 22, no. 114. Journal of Machine Learning Research, p. 1−43, 2021.","ista":"Ramezani-Kebrya A, Faghri F, Markov I, Aksenov V, Alistarh D-A, Roy DM. 2021. NUQSGD: Provably communication-efficient data-parallel SGD via nonuniform quantization. Journal of Machine Learning Research. 22(114), 1−43.","mla":"Ramezani-Kebrya, Ali, et al. “NUQSGD: Provably Communication-Efficient Data-Parallel SGD via Nonuniform Quantization.” <i>Journal of Machine Learning Research</i>, vol. 22, no. 114, Journal of Machine Learning Research, 2021, p. 1−43.","ama":"Ramezani-Kebrya A, Faghri F, Markov I, Aksenov V, Alistarh D-A, Roy DM. NUQSGD: Provably communication-efficient data-parallel SGD via nonuniform quantization. <i>Journal of Machine Learning Research</i>. 2021;22(114):1−43."},"page":"1−43","abstract":[{"lang":"eng","text":"As the size and complexity of models and datasets grow, so does the need for communication-efficient variants of stochastic gradient descent that can be deployed to perform parallel model training. One popular communication-compression method for data-parallel SGD is QSGD (Alistarh et al., 2017), which quantizes and encodes gradients to reduce communication costs. The baseline variant of QSGD provides strong theoretical guarantees, however, for practical purposes, the authors proposed a heuristic variant which we call QSGDinf, which demonstrated impressive empirical gains for distributed training of large neural networks. In this paper, we build on this work to propose a new gradient quantization scheme, and show that it has both stronger theoretical guarantees than QSGD, and matches and exceeds the empirical performance of the QSGDinf heuristic and of other compression methods."}],"type":"journal_article","oa_version":"Published Version","article_type":"original","date_created":"2021-06-20T22:01:33Z","author":[{"last_name":"Ramezani-Kebrya","first_name":"Ali","full_name":"Ramezani-Kebrya, Ali"},{"full_name":"Faghri, Fartash","first_name":"Fartash","last_name":"Faghri"},{"last_name":"Markov","first_name":"Ilya","full_name":"Markov, Ilya"},{"full_name":"Aksenov, Vitalii","first_name":"Vitalii","last_name":"Aksenov","id":"2980135A-F248-11E8-B48F-1D18A9856A87"},{"full_name":"Alistarh, Dan-Adrian","orcid":"0000-0003-3650-940X","id":"4A899BFC-F248-11E8-B48F-1D18A9856A87","first_name":"Dan-Adrian","last_name":"Alistarh"},{"full_name":"Roy, Daniel M.","first_name":"Daniel M.","last_name":"Roy"}],"status":"public","title":"NUQSGD: Provably communication-efficient data-parallel SGD via nonuniform quantization","_id":"9571"},{"language":[{"iso":"eng"}],"acknowledgement":"Peter Davies is supported by the European Union’s Horizon2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 754411.","publication_identifier":{"eissn":["1611-3349"],"isbn":["9783030795269"],"eisbn":["9783030795276"],"issn":["0302-9743"]},"ec_funded":1,"publication":"Structural Information and Communication Complexity","scopus_import":"1","user_id":"317138e5-6ab7-11ef-aa6d-ffef3953e345","date_updated":"2025-09-10T10:04:46Z","project":[{"_id":"260C2330-B435-11E9-9278-68D0E5697425","call_identifier":"H2020","name":"ISTplus - Postdoctoral Fellowships","grant_number":"754411"}],"citation":{"apa":"Alistarh, D.-A., &#38; Davies, P. (2021). Collecting coupons is faster with friends. In <i>Structural Information and Communication Complexity</i> (Vol. 12810, pp. 3–12). Wrocław, Poland: Springer Nature. <a href=\"https://doi.org/10.1007/978-3-030-79527-6_1\">https://doi.org/10.1007/978-3-030-79527-6_1</a>","chicago":"Alistarh, Dan-Adrian, and Peter Davies. “Collecting Coupons Is Faster with Friends.” In <i>Structural Information and Communication Complexity</i>, 12810:3–12. Springer Nature, 2021. <a href=\"https://doi.org/10.1007/978-3-030-79527-6_1\">https://doi.org/10.1007/978-3-030-79527-6_1</a>.","ieee":"D.-A. Alistarh and P. Davies, “Collecting coupons is faster with friends,” in <i>Structural Information and Communication Complexity</i>, Wrocław, Poland, 2021, vol. 12810, pp. 3–12.","short":"D.-A. Alistarh, P. Davies, in:, Structural Information and Communication Complexity, Springer Nature, 2021, pp. 3–12.","ista":"Alistarh D-A, Davies P. 2021. Collecting coupons is faster with friends. Structural Information and Communication Complexity. SIROCCO: International Colloquium on Structural Information and Communication Complexity, LNCS, vol. 12810, 3–12.","mla":"Alistarh, Dan-Adrian, and Peter Davies. “Collecting Coupons Is Faster with Friends.” <i>Structural Information and Communication Complexity</i>, vol. 12810, Springer Nature, 2021, pp. 3–12, doi:<a href=\"https://doi.org/10.1007/978-3-030-79527-6_1\">10.1007/978-3-030-79527-6_1</a>.","ama":"Alistarh D-A, Davies P. Collecting coupons is faster with friends. In: <i>Structural Information and Communication Complexity</i>. Vol 12810. Springer Nature; 2021:3-12. doi:<a href=\"https://doi.org/10.1007/978-3-030-79527-6_1\">10.1007/978-3-030-79527-6_1</a>"},"page":"3-12","oa_version":"Preprint","type":"conference","abstract":[{"text":"In this note, we introduce a distributed twist on the classic coupon collector problem: a set of m collectors wish to each obtain a set of n coupons; for this, they can each sample coupons uniformly at random, but can also meet in pairwise interactions, during which they can exchange coupons. By doing so, they hope to reduce the number of coupons that must be sampled by each collector in order to obtain a full set. This extension is natural when considering real-world manifestations of the coupon collector phenomenon, and has been remarked upon and studied empirically (Hayes and Hannigan 2006, Ahmad et al. 2014, Delmarcelle 2019).\r\n\r\nWe provide the first theoretical analysis for such a scenario. We find that “coupon collecting with friends” can indeed significantly reduce the number of coupons each collector must sample, and raises interesting connections to the more traditional variants of the problem. While our analysis is in most cases asymptotically tight, there are several open questions raised, regarding finer-grained analysis of both “coupon collecting with friends,” and of a long-studied variant of the original problem in which a collector requires multiple full sets of coupons.","lang":"eng"}],"_id":"9620","title":"Collecting coupons is faster with friends","date_created":"2021-07-01T11:04:43Z","status":"public","author":[{"first_name":"Dan-Adrian","last_name":"Alistarh","id":"4A899BFC-F248-11E8-B48F-1D18A9856A87","full_name":"Alistarh, Dan-Adrian","orcid":"0000-0003-3650-940X"},{"id":"11396234-BB50-11E9-B24C-90FCE5697425","first_name":"Peter","last_name":"Davies","orcid":"0000-0002-5646-9524","full_name":"Davies, Peter"}],"volume":12810,"file":[{"file_name":"Population_Coupon_Collector.pdf","date_updated":"2021-07-01T11:21:40Z","content_type":"application/pdf","file_size":319728,"file_id":"9621","access_level":"open_access","creator":"pdavies","relation":"main_file","checksum":"fe37fb9af3f5016c1084af9d6e7109bd","date_created":"2021-07-01T11:21:40Z"}],"doi":"10.1007/978-3-030-79527-6_1","isi":1,"external_id":{"isi":["001292788400001"]},"department":[{"_id":"DaAl"}],"month":"06","publisher":"Springer Nature","day":"20","oa":1,"fulldoi":"https://doi.org/10.1007/978-3-030-79527-6_1","file_date_updated":"2021-07-01T11:21:40Z","alternative_title":["LNCS"],"date_published":"2021-06-20T00:00:00Z","has_accepted_license":"1","conference":{"end_date":"2021-07-01","start_date":"2021-06-28","name":"SIROCCO: International Colloquium on Structural Information and Communication Complexity","location":"Wrocław, Poland"},"article_processing_charge":"No","year":"2021","quality_controlled":"1","publication_status":"published","intvolume":"     12810","ddc":["000"]},{"page":"87-105","citation":{"ieee":"D.-A. Alistarh, F. Ellen, and J. Rybicki, “Wait-free approximate agreement on graphs,” in <i>Structural Information and Communication Complexity</i>, Wrocław, Poland, 2021, vol. 12810, pp. 87–105.","ista":"Alistarh D-A, Ellen F, Rybicki J. 2021. Wait-free approximate agreement on graphs. Structural Information and Communication Complexity. SIROCCO: Structural Information and Communication Complexity, LNCS, vol. 12810, 87–105.","short":"D.-A. Alistarh, F. Ellen, J. Rybicki, in:, Structural Information and Communication Complexity, Springer Nature, 2021, pp. 87–105.","ama":"Alistarh D-A, Ellen F, Rybicki J. Wait-free approximate agreement on graphs. In: <i>Structural Information and Communication Complexity</i>. Vol 12810. Springer Nature; 2021:87-105. doi:<a href=\"https://doi.org/10.1007/978-3-030-79527-6_6\">10.1007/978-3-030-79527-6_6</a>","mla":"Alistarh, Dan-Adrian, et al. “Wait-Free Approximate Agreement on Graphs.” <i>Structural Information and Communication Complexity</i>, vol. 12810, Springer Nature, 2021, pp. 87–105, doi:<a href=\"https://doi.org/10.1007/978-3-030-79527-6_6\">10.1007/978-3-030-79527-6_6</a>.","apa":"Alistarh, D.-A., Ellen, F., &#38; Rybicki, J. (2021). Wait-free approximate agreement on graphs. In <i>Structural Information and Communication Complexity</i> (Vol. 12810, pp. 87–105). Wrocław, Poland: Springer Nature. <a href=\"https://doi.org/10.1007/978-3-030-79527-6_6\">https://doi.org/10.1007/978-3-030-79527-6_6</a>","chicago":"Alistarh, Dan-Adrian, Faith Ellen, and Joel Rybicki. “Wait-Free Approximate Agreement on Graphs.” In <i>Structural Information and Communication Complexity</i>, 12810:87–105. Springer Nature, 2021. <a href=\"https://doi.org/10.1007/978-3-030-79527-6_6\">https://doi.org/10.1007/978-3-030-79527-6_6</a>."},"date_updated":"2026-04-16T09:26:11Z","user_id":"ba8df636-2132-11f1-aed0-ed93e2281fdd","date_created":"2021-08-08T22:01:29Z","author":[{"id":"4A899BFC-F248-11E8-B48F-1D18A9856A87","first_name":"Dan-Adrian","last_name":"Alistarh","full_name":"Alistarh, Dan-Adrian","orcid":"0000-0003-3650-940X"},{"first_name":"Faith","last_name":"Ellen","full_name":"Ellen, Faith"},{"first_name":"Joel","last_name":"Rybicki","id":"334EFD2E-F248-11E8-B48F-1D18A9856A87","full_name":"Rybicki, Joel","orcid":"0000-0002-6432-6646"}],"status":"public","title":"Wait-free approximate agreement on graphs","_id":"9823","abstract":[{"lang":"eng","text":"Approximate agreement is one of the few variants of consensus that can be solved in a wait-free manner in asynchronous systems where processes communicate by reading and writing to shared memory. In this work, we consider a natural generalisation of approximate agreement on arbitrary undirected connected graphs. Each process is given a vertex of the graph as input and, if non-faulty, must output a vertex such that\r\nall the outputs are within distance 1 of one another, and\r\n\r\neach output value lies on a shortest path between two input values.\r\n\r\nFrom prior work, it is known that there is no wait-free algorithm among   𝑛≥3  processes for this problem on any cycle of length   𝑐≥4 , by reduction from 2-set agreement (Castañeda et al. 2018).\r\n\r\nIn this work, we investigate the solvability and complexity of this task on general graphs. We give a new, direct proof of the impossibility of approximate agreement on cycles of length   𝑐≥4 , via a generalisation of Sperner’s Lemma to convex polygons. We also extend the reduction from 2-set agreement to a larger class of graphs, showing that approximate agreement on these graphs is unsolvable. On the positive side, we present a wait-free algorithm for a class of graphs that properly contains the class of chordal graphs."}],"oa_version":"Preprint","type":"conference","language":[{"iso":"eng"}],"scopus_import":"1","publication":"Structural Information and Communication Complexity","publication_identifier":{"eissn":["1611-3349"],"isbn":["9783030795269"],"issn":["0302-9743"]},"article_processing_charge":"No","conference":{"location":"Wrocław, Poland","name":"SIROCCO: Structural Information and Communication Complexity","start_date":"2021-06-28","end_date":"2021-07-01"},"date_published":"2021-06-20T00:00:00Z","alternative_title":["LNCS"],"main_file_link":[{"url":"https://arxiv.org/abs/2103.08949","open_access":"1"}],"arxiv":1,"fulldoi":"https://doi.org/10.1007/978-3-030-79527-6_6","intvolume":"     12810","publication_status":"published","year":"2021","quality_controlled":"1","isi":1,"doi":"10.1007/978-3-030-79527-6_6","volume":12810,"oa":1,"day":"20","publisher":"Springer Nature","department":[{"_id":"DaAl"}],"month":"06","external_id":{"isi":["001292788400006"],"arxiv":["2103.08949"]}},{"oa_version":"Submitted Version","type":"journal_article","article_type":"original","abstract":[{"text":"The Nearest neighbour search (NNS) is a fundamental problem in many application domains dealing with multidimensional data. In a concurrent setting, where dynamic modifications are allowed, a linearizable implementation of the NNS is highly desirable.This paper introduces the LockFree-kD-tree (LFkD-tree ): a lock-free concurrent kD-tree, which implements an abstract data type (ADT) that provides the operations Add, Remove, Contains, and NNS. Our implementation is linearizable. The operations in the LFkD-tree use single-word read and compare-and-swap (Image 1 ) atomic primitives, which are readily supported on available multi-core processors. We experimentally evaluate the LFkD-tree using several benchmarks comprising real-world and synthetic datasets. The experiments show that the presented design is scalable and achieves significant speed-up compared to the implementations of an existing sequential kD-tree and a recently proposed multidimensional indexing structure, PH-tree.","lang":"eng"}],"title":"Concurrent linearizable nearest neighbour search in LockFree-kD-tree","_id":"9827","date_created":"2021-08-08T22:01:31Z","status":"public","author":[{"orcid":"0000-0002-2742-4028","full_name":"Chatterjee, Bapi","last_name":"Chatterjee","first_name":"Bapi","id":"3C41A08A-F248-11E8-B48F-1D18A9856A87"},{"last_name":"Walulya","first_name":"Ivan","full_name":"Walulya, Ivan"},{"full_name":"Tsigas, Philippas","last_name":"Tsigas","first_name":"Philippas"}],"date_updated":"2024-10-09T21:00:45Z","user_id":"4359f0d1-fa6c-11eb-b949-802e58b17ae8","citation":{"chicago":"Chatterjee, Bapi, Ivan Walulya, and Philippas Tsigas. “Concurrent Linearizable Nearest Neighbour Search in LockFree-KD-Tree.” <i>Theoretical Computer Science</i>. Elsevier, 2021. <a href=\"https://doi.org/10.1016/j.tcs.2021.06.041\">https://doi.org/10.1016/j.tcs.2021.06.041</a>.","apa":"Chatterjee, B., Walulya, I., &#38; Tsigas, P. (2021). Concurrent linearizable nearest neighbour search in LockFree-kD-tree. <i>Theoretical Computer Science</i>. Elsevier. <a href=\"https://doi.org/10.1016/j.tcs.2021.06.041\">https://doi.org/10.1016/j.tcs.2021.06.041</a>","ama":"Chatterjee B, Walulya I, Tsigas P. Concurrent linearizable nearest neighbour search in LockFree-kD-tree. <i>Theoretical Computer Science</i>. 2021;886:27-48. doi:<a href=\"https://doi.org/10.1016/j.tcs.2021.06.041\">10.1016/j.tcs.2021.06.041</a>","mla":"Chatterjee, Bapi, et al. “Concurrent Linearizable Nearest Neighbour Search in LockFree-KD-Tree.” <i>Theoretical Computer Science</i>, vol. 886, Elsevier, 2021, pp. 27–48, doi:<a href=\"https://doi.org/10.1016/j.tcs.2021.06.041\">10.1016/j.tcs.2021.06.041</a>.","short":"B. Chatterjee, I. Walulya, P. Tsigas, Theoretical Computer Science 886 (2021) 27–48.","ieee":"B. Chatterjee, I. Walulya, and P. Tsigas, “Concurrent linearizable nearest neighbour search in LockFree-kD-tree,” <i>Theoretical Computer Science</i>, vol. 886. Elsevier, pp. 27–48, 2021.","ista":"Chatterjee B, Walulya I, Tsigas P. 2021. Concurrent linearizable nearest neighbour search in LockFree-kD-tree. Theoretical Computer Science. 886, 27–48."},"page":"27-48","publication_identifier":{"issn":["0304-3975"]},"publication":"Theoretical Computer Science","scopus_import":"1","language":[{"iso":"eng"}],"publication_status":"published","quality_controlled":"1","year":"2021","intvolume":"       886","fulldoi":"https://doi.org/10.1016/j.tcs.2021.06.041","main_file_link":[{"url":"https://publications.lib.chalmers.se/records/fulltext/232185/232185.pdf","open_access":"1"}],"date_published":"2021-09-13T00:00:00Z","article_processing_charge":"No","keyword":["Concurrent data structure","kD-tree","Nearest neighbor search","Similarity search","Lock-free","Linearizability"],"external_id":{"isi":["000694718900004"]},"publisher":"Elsevier","month":"09","department":[{"_id":"DaAl"}],"day":"13","oa":1,"corr_author":"1","volume":886,"doi":"10.1016/j.tcs.2021.06.041","isi":1},{"ec_funded":1,"acknowledgement":"This work is partially supported by a Weizmann-UK Making Connections Grant, the Centre for Discrete Mathematics and its Applications (DIMAP), IBM Faculty Award, EPSRC award EP/V01305X/1, European Research Council (ERC) Grant No. 949083, the Minerva foundation with funding from the Federal German Ministry for Education and Research No. 713238, and the European Union’s Horizon 2020 programme under the Marie Skłodowska-Curie grant agreement No 754411.","publication_identifier":{"isbn":["9781450385480"]},"scopus_import":"1","publication":"Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing","language":[{"iso":"eng"}],"abstract":[{"lang":"eng","text":"In this paper, we study the power and limitations of component-stable algorithms in the low-space model of Massively Parallel Computation (MPC). Recently Ghaffari, Kuhn and Uitto (FOCS 2019) introduced the class of component-stable low-space MPC algorithms, which are, informally, defined as algorithms for which the outputs reported by the nodes in different connected components are required to be independent. This very natural notion was introduced to capture most (if not all) of the known efficient MPC algorithms to date, and it was the first general class of MPC algorithms for which one can show non-trivial conditional lower bounds. In this paper we enhance the framework of component-stable algorithms and investigate its effect on the complexity of randomized and deterministic low-space MPC. Our key contributions include: 1) We revise and formalize the lifting approach of Ghaffari, Kuhn and Uitto. This requires a very delicate amendment of the notion of component stability, which allows us to fill in gaps in the earlier arguments. 2) We also extend the framework to obtain conditional lower bounds for deterministic algorithms and fine-grained lower bounds that depend on the maximum degree Δ. 3) We demonstrate a collection of natural graph problems for which non-component-stable algorithms break the conditional lower bound obtained for component-stable algorithms. This implies that, for both deterministic and randomized algorithms, component-stable algorithms are conditionally weaker than the non-component-stable ones.\r\n\r\nAltogether our results imply that component-stability might limit the computational power of the low-space MPC model, paving the way for improved upper bounds that escape the conditional lower bound setting of Ghaffari, Kuhn, and Uitto."}],"type":"conference","oa_version":"Submitted Version","date_created":"2021-08-17T18:11:16Z","author":[{"full_name":"Czumaj, Artur","first_name":"Artur","last_name":"Czumaj"},{"orcid":"0000-0002-5646-9524","full_name":"Davies, Peter","first_name":"Peter","last_name":"Davies","id":"11396234-BB50-11E9-B24C-90FCE5697425"},{"full_name":"Parter, Merav","first_name":"Merav","last_name":"Parter"}],"status":"public","_id":"9933","title":"Component stability in low-space massively parallel computation","user_id":"4359f0d1-fa6c-11eb-b949-802e58b17ae8","date_updated":"2025-04-14T07:43:49Z","project":[{"_id":"260C2330-B435-11E9-9278-68D0E5697425","call_identifier":"H2020","name":"ISTplus - Postdoctoral Fellowships","grant_number":"754411"}],"citation":{"apa":"Czumaj, A., Davies, P., &#38; Parter, M. (2021). Component stability in low-space massively parallel computation. In <i>Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing</i> (pp. 481–491). Virtual, Italy: Association for Computing Machinery. <a href=\"https://doi.org/10.1145/3465084.3467903\">https://doi.org/10.1145/3465084.3467903</a>","chicago":"Czumaj, Artur, Peter Davies, and Merav Parter. “Component Stability in Low-Space Massively Parallel Computation.” In <i>Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing</i>, 481–491. Association for Computing Machinery, 2021. <a href=\"https://doi.org/10.1145/3465084.3467903\">https://doi.org/10.1145/3465084.3467903</a>.","short":"A. Czumaj, P. Davies, M. Parter, in:, Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing, Association for Computing Machinery, 2021, pp. 481–491.","ieee":"A. Czumaj, P. Davies, and M. Parter, “Component stability in low-space massively parallel computation,” in <i>Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing</i>, Virtual, Italy, 2021, pp. 481–491.","ista":"Czumaj A, Davies P, Parter M. 2021. Component stability in low-space massively parallel computation. Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing. PODC: Principles of Distributed Computing, 481–491.","mla":"Czumaj, Artur, et al. “Component Stability in Low-Space Massively Parallel Computation.” <i>Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing</i>, Association for Computing Machinery, 2021, pp. 481–491, doi:<a href=\"https://doi.org/10.1145/3465084.3467903\">10.1145/3465084.3467903</a>.","ama":"Czumaj A, Davies P, Parter M. Component stability in low-space massively parallel computation. In: <i>Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing</i>. Association for Computing Machinery; 2021:481–491. doi:<a href=\"https://doi.org/10.1145/3465084.3467903\">10.1145/3465084.3467903</a>"},"page":"481–491","department":[{"_id":"DaAl"}],"month":"07","publisher":"Association for Computing Machinery","external_id":{"arxiv":["2106.01880"],"isi":["000744439800049"]},"oa":1,"day":"21","doi":"10.1145/3465084.3467903","isi":1,"year":"2021","quality_controlled":"1","publication_status":"published","arxiv":1,"main_file_link":[{"open_access":"1","url":"https://arxiv.org/abs/2106.01880"}],"fulldoi":"https://doi.org/10.1145/3465084.3467903","article_processing_charge":"No","date_published":"2021-07-21T00:00:00Z","conference":{"name":"PODC: Principles of Distributed Computing","end_date":"2021-07-30","start_date":"2021-07-26","location":"Virtual, Italy"}},{"language":[{"iso":"eng"}],"acknowledgement":"This work is partially supported by a Weizmann-UK Making Connections Grant, the Centre for Discrete Mathematics and its Applications (DIMAP), IBM Faculty Award, EPSRC award EP/V01305X/1, European Research Council (ERC) Grant No. 949083, the Minerva foundation with funding from the Federal German Ministry for Education and Research No. 713238, and the European Union’s Horizon 2020 programme under the Marie Skłodowska-Curie grant agreement No 754411.","ec_funded":1,"publication_identifier":{"isbn":["978-1-4503-8548-0"]},"scopus_import":"1","publication":"Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing","date_updated":"2025-04-14T07:43:49Z","user_id":"4359f0d1-fa6c-11eb-b949-802e58b17ae8","page":"469–479","citation":{"apa":"Czumaj, A., Davies, P., &#38; Parter, M. (2021). Improved deterministic (Δ+1) coloring in low-space MPC. In <i>Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing</i> (pp. 469–479). Virtual, Italy: Association for Computing Machinery. <a href=\"https://doi.org/10.1145/3465084.3467937\">https://doi.org/10.1145/3465084.3467937</a>","chicago":"Czumaj, Artur, Peter Davies, and Merav Parter. “Improved Deterministic (Δ+1) Coloring in Low-Space MPC.” In <i>Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing</i>, 469–479. Association for Computing Machinery, 2021. <a href=\"https://doi.org/10.1145/3465084.3467937\">https://doi.org/10.1145/3465084.3467937</a>.","ieee":"A. Czumaj, P. Davies, and M. Parter, “Improved deterministic (Δ+1) coloring in low-space MPC,” in <i>Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing</i>, Virtual, Italy, 2021, pp. 469–479.","ista":"Czumaj A, Davies P, Parter M. 2021. Improved deterministic (Δ+1) coloring in low-space MPC. Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing. PODC: Symposium on Principles of Distributed Computing, 469–479.","short":"A. Czumaj, P. Davies, M. Parter, in:, Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing, Association for Computing Machinery, 2021, pp. 469–479.","mla":"Czumaj, Artur, et al. “Improved Deterministic (Δ+1) Coloring in Low-Space MPC.” <i>Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing</i>, Association for Computing Machinery, 2021, pp. 469–479, doi:<a href=\"https://doi.org/10.1145/3465084.3467937\">10.1145/3465084.3467937</a>.","ama":"Czumaj A, Davies P, Parter M. Improved deterministic (Δ+1) coloring in low-space MPC. In: <i>Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing</i>. Association for Computing Machinery; 2021:469–479. doi:<a href=\"https://doi.org/10.1145/3465084.3467937\">10.1145/3465084.3467937</a>"},"project":[{"grant_number":"754411","name":"ISTplus - Postdoctoral Fellowships","call_identifier":"H2020","_id":"260C2330-B435-11E9-9278-68D0E5697425"}],"abstract":[{"lang":"eng","text":"We present a deterministic O(log log log n)-round low-space Massively Parallel Computation (MPC) algorithm for the classical problem of (Δ+1)-coloring on n-vertex graphs. In this model, every machine has sublinear local space of size n^φ for any arbitrary constant φ \\in (0,1). Our algorithm works under the relaxed setting where each machine is allowed to perform exponential local computations, while respecting the n^φ space and bandwidth limitations.\r\n\r\nOur key technical contribution is a novel derandomization of the ingenious (Δ+1)-coloring local algorithm by Chang-Li-Pettie (STOC 2018, SIAM J. Comput. 2020). The Chang-Li-Pettie algorithm runs in T_local =poly(loglog n) rounds, which sets the state-of-the-art randomized round complexity for the problem in the local model. Our derandomization employs a combination of tools, notably pseudorandom generators (PRG) and bounded-independence hash functions.\r\n\r\nThe achieved round complexity of O(logloglog n) rounds matches the bound of log(T_local ), which currently serves an upper bound barrier for all known randomized algorithms for locally-checkable problems in this model. Furthermore, no deterministic sublogarithmic low-space MPC algorithms for the (Δ+1)-coloring problem have been known before."}],"type":"conference","oa_version":"Submitted Version","status":"public","author":[{"full_name":"Czumaj, Artur","first_name":"Artur","last_name":"Czumaj"},{"orcid":"0000-0002-5646-9524","full_name":"Davies, Peter","id":"11396234-BB50-11E9-B24C-90FCE5697425","last_name":"Davies","first_name":"Peter"},{"full_name":"Parter, Merav","last_name":"Parter","first_name":"Merav"}],"date_created":"2021-08-17T18:14:15Z","title":"Improved deterministic (Δ+1) coloring in low-space MPC","_id":"9935","doi":"10.1145/3465084.3467937","isi":1,"publisher":"Association for Computing Machinery","department":[{"_id":"DaAl"}],"month":"07","external_id":{"isi":["000744439800048"]},"oa":1,"day":"21","main_file_link":[{"url":"http://wrap.warwick.ac.uk/153753","open_access":"1"}],"fulldoi":"https://doi.org/10.1145/3465084.3467937","article_processing_charge":"No","conference":{"start_date":"2021-07-26","end_date":"2021-07-30","name":"PODC: Symposium on Principles of Distributed Computing","location":"Virtual, Italy"},"date_published":"2021-07-21T00:00:00Z","publication_status":"published","year":"2021","quality_controlled":"1"},{"language":[{"iso":"eng"}],"publication":"Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing","scopus_import":"1","acknowledgement":"We would like to thank Rati Gelashvili for very useful discussions, and the PODC anonymous reviewers for their careful reading of our paper, and for their useful remarks. This work is partially supported by the Polish National Science Center (NCN) grant UMO2017/25/B/ST6/02010.","publication_identifier":{"isbn":["9781450385480"]},"page":"55-65","citation":{"ama":"Alistarh D-A, Töpfer M, Uznański P. Comparison dynamics in population protocols. In: <i>Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing</i>. Association for Computing Machinery; 2021:55-65. doi:<a href=\"https://doi.org/10.1145/3465084.3467915\">10.1145/3465084.3467915</a>","mla":"Alistarh, Dan-Adrian, et al. “Comparison Dynamics in Population Protocols.” <i>Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing</i>, Association for Computing Machinery, 2021, pp. 55–65, doi:<a href=\"https://doi.org/10.1145/3465084.3467915\">10.1145/3465084.3467915</a>.","ista":"Alistarh D-A, Töpfer M, Uznański P. 2021. Comparison dynamics in population protocols. Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing. PODC: Symposium on Principles of Distributed Computing, 55–65.","ieee":"D.-A. Alistarh, M. Töpfer, and P. Uznański, “Comparison dynamics in population protocols,” in <i>Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing</i>, Virtual, Italy, 2021, pp. 55–65.","short":"D.-A. Alistarh, M. Töpfer, P. Uznański, in:, Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing, Association for Computing Machinery, 2021, pp. 55–65.","chicago":"Alistarh, Dan-Adrian, Martin Töpfer, and Przemysław Uznański. “Comparison Dynamics in Population Protocols.” In <i>Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing</i>, 55–65. Association for Computing Machinery, 2021. <a href=\"https://doi.org/10.1145/3465084.3467915\">https://doi.org/10.1145/3465084.3467915</a>.","apa":"Alistarh, D.-A., Töpfer, M., &#38; Uznański, P. (2021). Comparison dynamics in population protocols. In <i>Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing</i> (pp. 55–65). Virtual, Italy: Association for Computing Machinery. <a href=\"https://doi.org/10.1145/3465084.3467915\">https://doi.org/10.1145/3465084.3467915</a>"},"date_updated":"2023-08-11T10:56:04Z","user_id":"4359f0d1-fa6c-11eb-b949-802e58b17ae8","title":"Comparison dynamics in population protocols","_id":"9951","status":"public","date_created":"2021-08-22T22:01:20Z","author":[{"full_name":"Alistarh, Dan-Adrian","orcid":"0000-0003-3650-940X","id":"4A899BFC-F248-11E8-B48F-1D18A9856A87","first_name":"Dan-Adrian","last_name":"Alistarh"},{"full_name":"Töpfer, Martin","first_name":"Martin","last_name":"Töpfer","id":"4B865388-F248-11E8-B48F-1D18A9856A87"},{"full_name":"Uznański, Przemysław","last_name":"Uznański","first_name":"Przemysław"}],"oa_version":"None","type":"conference","abstract":[{"lang":"eng","text":"There has recently been a surge of interest in the computational and complexity properties of the population model, which assumes n anonymous, computationally-bounded nodes, interacting at random, with the goal of jointly computing global predicates. Significant work has gone towards investigating majority or consensus dynamics in this model: that is, assuming that every node is initially in one of two states X or Y, determine which state had higher initial count.\r\n\r\nIn this paper, we consider a natural generalization of majority/consensus, which we call comparison : in its simplest formulation, we are given two baseline states, X and Y, present in any initial configuration in fixed, but possibly small counts. One of these states has higher count than the other: we will assume |X_0| > C |Y_0| for some constant C > 1. The challenge is to design a protocol by which nodes can quickly and reliably decide on which of the baseline states X_0 and Y_0 has higher initial count. We begin by analyzing a simple and general dynamics solving the above comparison problem, which uses O( log n ) states per node, and converges in O(log n) (parallel) time, with high probability, to a state where the whole population votes on opinions X or Y at rates proportional to the initial concentrations of |X_0| vs. |Y_0|. We then describe how this procedure can be bootstrapped to solve comparison, i.e. have every node in the population reach the \"correct'' decision, with probability 1 - o(1), at the cost of O (log log n) additional states. Further, we prove that this dynamics is self-stabilizing, in the sense that it converges to the correct decision from arbitrary initial states, and leak-robust, in the sense that it can withstand spurious faulty reactions, which are known to occur in practical implementations of population protocols. Our analysis is based on a new martingale concentration result relating the discrete-time evolution of a population protocol to its expected (steady-state) analysis, which should be a useful tool when analyzing opinion dynamics and epidemic dissemination in the population model."}],"isi":1,"doi":"10.1145/3465084.3467915","day":"21","external_id":{"isi":["000744439800005"]},"publisher":"Association for Computing Machinery","department":[{"_id":"DaAl"}],"month":"07","conference":{"location":"Virtual, Italy","name":"PODC: Symposium on Principles of Distributed Computing","end_date":"2021-07-30","start_date":"2021-07-26"},"date_published":"2021-07-21T00:00:00Z","article_processing_charge":"No","fulldoi":"https://doi.org/10.1145/3465084.3467915","publication_status":"published","quality_controlled":"1","year":"2021"},{"publication":"Annual ACM Symposium on Parallelism in Algorithms and Architectures","scopus_import":"1","ec_funded":1,"publication_identifier":{"isbn":["9781450380706"]},"acknowledgement":"We thank Orr Fischer, Juho Hirvonen, and Tuomo Lempiäinen for valuable discussions. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 840605.","language":[{"iso":"eng"}],"_id":"9678","title":"Efficient load-balancing through distributed token dropping","status":"public","date_created":"2021-07-18T22:01:22Z","author":[{"full_name":"Brandt, Sebastian","last_name":"Brandt","first_name":"Sebastian"},{"last_name":"Keller","first_name":"Barbara","full_name":"Keller, Barbara"},{"id":"334EFD2E-F248-11E8-B48F-1D18A9856A87","first_name":"Joel","last_name":"Rybicki","full_name":"Rybicki, Joel","orcid":"0000-0002-6432-6646"},{"full_name":"Suomela, Jukka","last_name":"Suomela","first_name":"Jukka"},{"full_name":"Uitto, Jara","first_name":"Jara","last_name":"Uitto"}],"type":"conference","oa_version":"Preprint","abstract":[{"lang":"eng","text":"We introduce a new graph problem, the token dropping game, and we show how to solve it efficiently in a distributed setting. We use the token dropping game as a tool to design an efficient distributed algorithm for stable orientations and more generally for locally optimal semi-matchings. The prior work by Czygrinow et al. (DISC 2012) finds a stable orientation in O(Δ^5) rounds in graphs of maximum degree Δ, while we improve it to O(Δ^4) and also prove a lower bound of Ω(Δ). For the more general problem of locally optimal semi-matchings, the prior upper bound is O(S^5) and our new algorithm runs in O(C · S^4) rounds, which is an improvement for C = o(S); here C and S are the maximum degrees of customers and servers, respectively."}],"related_material":{"record":[{"status":"public","relation":"earlier_version","id":"15074"}]},"project":[{"_id":"26A5D39A-B435-11E9-9278-68D0E5697425","name":"Coordination in constrained and natural distributed systems","call_identifier":"H2020","grant_number":"840605"}],"page":"129-139","citation":{"apa":"Brandt, S., Keller, B., Rybicki, J., Suomela, J., &#38; Uitto, J. (2021). Efficient load-balancing through distributed token dropping. In <i>Annual ACM Symposium on Parallelism in Algorithms and Architectures</i> (pp. 129–139).  Virtual Event, United States: Association for Computing Machinery. <a href=\"https://doi.org/10.1145/3409964.3461785\">https://doi.org/10.1145/3409964.3461785</a>","chicago":"Brandt, Sebastian, Barbara Keller, Joel Rybicki, Jukka Suomela, and Jara Uitto. “Efficient Load-Balancing through Distributed Token Dropping.” In <i>Annual ACM Symposium on Parallelism in Algorithms and Architectures</i>, 129–39. Association for Computing Machinery, 2021. <a href=\"https://doi.org/10.1145/3409964.3461785\">https://doi.org/10.1145/3409964.3461785</a>.","ista":"Brandt S, Keller B, Rybicki J, Suomela J, Uitto J. 2021. Efficient load-balancing through distributed token dropping. Annual ACM Symposium on Parallelism in Algorithms and Architectures. SPAA: Symposium on Parallelism in Algorithms and Architectures , 129–139.","ieee":"S. Brandt, B. Keller, J. Rybicki, J. Suomela, and J. Uitto, “Efficient load-balancing through distributed token dropping,” in <i>Annual ACM Symposium on Parallelism in Algorithms and Architectures</i>,  Virtual Event, United States, 2021, pp. 129–139.","short":"S. Brandt, B. Keller, J. Rybicki, J. Suomela, J. Uitto, in:, Annual ACM Symposium on Parallelism in Algorithms and Architectures, Association for Computing Machinery, 2021, pp. 129–139.","ama":"Brandt S, Keller B, Rybicki J, Suomela J, Uitto J. Efficient load-balancing through distributed token dropping. In: <i>Annual ACM Symposium on Parallelism in Algorithms and Architectures</i>. Association for Computing Machinery; 2021:129-139. doi:<a href=\"https://doi.org/10.1145/3409964.3461785\">10.1145/3409964.3461785</a>","mla":"Brandt, Sebastian, et al. “Efficient Load-Balancing through Distributed Token Dropping.” <i>Annual ACM Symposium on Parallelism in Algorithms and Architectures</i>, Association for Computing Machinery, 2021, pp. 129–39, doi:<a href=\"https://doi.org/10.1145/3409964.3461785\">10.1145/3409964.3461785</a>."},"user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","date_updated":"2026-07-07T06:21:32Z","day":"06","oa":1,"external_id":{"arxiv":["2005.07761"]},"month":"07","department":[{"_id":"DaAl"}],"publisher":"Association for Computing Machinery","doi":"10.1145/3409964.3461785","quality_controlled":"1","year":"2021","publication_status":"published","date_published":"2021-07-06T00:00:00Z","conference":{"location":" Virtual Event, United States","end_date":"2021-07-08","start_date":"2021-07-06","name":"SPAA: Symposium on Parallelism in Algorithms and Architectures "},"article_processing_charge":"No","fulldoi":"https://doi.org/10.1145/3409964.3461785","das_tickbox":"1","arxiv":1,"main_file_link":[{"open_access":"1","url":"https://arxiv.org/abs/2005.07761"}]},{"abstract":[{"text":"We study efficient distributed algorithms for the fundamental problem of principal component analysis and leading eigenvector computation on the sphere, when the data are randomly distributed among a set of computational nodes. We propose a new quantized variant of Riemannian gradient descent to solve this problem, and prove that the algorithm converges with high probability under a set of necessary spherical-convexity properties. We give bounds on the number of bits transmitted by the algorithm under common initialization schemes, and investigate the dependency on the problem dimension in each case.","lang":"eng"}],"type":"conference","oa_version":"Published Version","status":"public","date_created":"2022-06-19T22:01:58Z","author":[{"last_name":"Alimisis","first_name":"Foivos","full_name":"Alimisis, Foivos"},{"full_name":"Davies, Peter","orcid":"0000-0002-5646-9524","id":"11396234-BB50-11E9-B24C-90FCE5697425","first_name":"Peter","last_name":"Davies"},{"full_name":"Vandereycken, Bart","first_name":"Bart","last_name":"Vandereycken"},{"id":"4A899BFC-F248-11E8-B48F-1D18A9856A87","last_name":"Alistarh","first_name":"Dan-Adrian","full_name":"Alistarh, Dan-Adrian","orcid":"0000-0003-3650-940X"}],"_id":"11452","title":"Distributed principal component analysis with limited communication","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","date_updated":"2026-07-08T05:44:33Z","project":[{"call_identifier":"H2020","name":"Elastic Coordination for Scalable Machine Learning","_id":"268A44D6-B435-11E9-9278-68D0E5697425","grant_number":"805223"},{"grant_number":"754411","call_identifier":"H2020","name":"ISTplus - Postdoctoral Fellowships","_id":"260C2330-B435-11E9-9278-68D0E5697425"}],"page":"2823-2834","citation":{"short":"F. Alimisis, P. Davies, B. Vandereycken, D.-A. Alistarh, in:, 35th Conference on Neural Information Processing Systems, Neural Information Processing Systems Foundation, 2021, pp. 2823–2834.","ista":"Alimisis F, Davies P, Vandereycken B, Alistarh D-A. 2021. Distributed principal component analysis with limited communication. 35th Conference on Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems, Advances in Neural Information Processing Systems, vol. 4, 2823–2834.","ieee":"F. Alimisis, P. Davies, B. Vandereycken, and D.-A. Alistarh, “Distributed principal component analysis with limited communication,” in <i>35th Conference on Neural Information Processing Systems</i>, Virtual, Online, 2021, vol. 4, pp. 2823–2834.","mla":"Alimisis, Foivos, et al. “Distributed Principal Component Analysis with Limited Communication.” <i>35th Conference on Neural Information Processing Systems</i>, vol. 4, Neural Information Processing Systems Foundation, 2021, pp. 2823–34.","ama":"Alimisis F, Davies P, Vandereycken B, Alistarh D-A. Distributed principal component analysis with limited communication. In: <i>35th Conference on Neural Information Processing Systems</i>. Vol 4. Neural Information Processing Systems Foundation; 2021:2823-2834.","apa":"Alimisis, F., Davies, P., Vandereycken, B., &#38; Alistarh, D.-A. (2021). Distributed principal component analysis with limited communication. In <i>35th Conference on Neural Information Processing Systems</i> (Vol. 4, pp. 2823–2834). Virtual, Online: Neural Information Processing Systems Foundation.","chicago":"Alimisis, Foivos, Peter Davies, Bart Vandereycken, and Dan-Adrian Alistarh. “Distributed Principal Component Analysis with Limited Communication.” In <i>35th Conference on Neural Information Processing Systems</i>, 4:2823–34. Neural Information Processing Systems Foundation, 2021."},"ec_funded":1,"publication_identifier":{"issn":["1049-5258"],"isbn":["9781713845393"]},"acknowledgement":"We would like to thank the anonymous reviewers for helpful comments and suggestions. We also thank Aurelien Lucchi and Antonio Orvieto for fruitful discussions at an early stage of this work. FA is partially supported by the SNSF under research project No. 192363 and conducted part of this work while at IST Austria under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 805223 ScaleML). PD partly conducted this work while at IST Austria and was supported by the European Union’s Horizon 2020 programme under the Marie Skłodowska-Curie grant agreement No. 754411.","scopus_import":"1","publication":"35th Conference on Neural Information Processing Systems","language":[{"iso":"eng"}],"intvolume":"         4","quality_controlled":"1","year":"2021","publication_status":"published","ddc":["000"],"arxiv":1,"main_file_link":[{"open_access":"1","url":"https://proceedings.neurips.cc/paper/2021/file/1680e9fa7b4dd5d62ece800239bb53bd-Paper.pdf"}],"das_tickbox":"1","article_processing_charge":"No","date_published":"2021-12-01T00:00:00Z","alternative_title":["Advances in Neural Information Processing Systems"],"conference":{"location":"Virtual, Online","name":"NeurIPS: Neural Information Processing Systems","start_date":"2021-12-06","end_date":"2021-12-14"},"department":[{"_id":"DaAl"}],"month":"12","publisher":"Neural Information Processing Systems Foundation","external_id":{"arxiv":["2110.14391"]},"oa":1,"corr_author":"1","day":"01","volume":4},{"day":"28","oa":1,"external_id":{"arxiv":["1703.01859"]},"publisher":"ACM","department":[{"_id":"DaAl"}],"month":"01","article_number":"13","volume":68,"doi":"10.1145/3446383","publication_status":"published","year":"2021","quality_controlled":"1","intvolume":"        68","date_published":"2021-01-28T00:00:00Z","article_processing_charge":"No","keyword":["Artificial Intelligence","Hardware and Architecture","Information Systems","Control and Systems Engineering","Software"],"fulldoi":"https://doi.org/10.1145/3446383","main_file_link":[{"open_access":"1","url":"https://doi.org/10.48550/arXiv.1703.01859"}],"arxiv":1,"publication":"Journal of the ACM","issue":"2","publication_identifier":{"issn":["0004-5411"],"eissn":["1557-735X"]},"language":[{"iso":"eng"}],"title":"Exploiting spontaneous transmissions for broadcasting and leader election in radio networks","_id":"15267","status":"public","date_created":"2024-04-03T07:41:46Z","author":[{"full_name":"Czumaj, Artur","last_name":"Czumaj","first_name":"Artur"},{"orcid":"0000-0002-5646-9524","full_name":"Davies, Peter","id":"11396234-BB50-11E9-B24C-90FCE5697425","last_name":"Davies","first_name":"Peter"}],"type":"journal_article","oa_version":"Preprint","article_type":"original","abstract":[{"lang":"eng","text":"We study two fundamental communication primitives: broadcasting and leader election in the classical model of multi-hop radio networks with unknown topology and without collision detection mechanisms. It has been known for almost 20 years that in undirected networks with n nodes and diameter D, randomized broadcasting requires Ω(D log n/D + log2 n) rounds, assuming that uninformed nodes are not allowed to communicate (until they are informed). Only very recently, Haeupler and Wajc (PODC'2016) showed that this bound can be improved for the model with spontaneous transmissions, providing an O(D log n log log n/log D + logO(1) n)-time broadcasting algorithm. In this article, we give a new and faster algorithm that completes broadcasting in O(D log n/log D + logO(1) n) time, succeeding with high probability. This yields the first optimal O(D)-time broadcasting algorithm whenever n is polynomial in D.\r\n\r\nFurthermore, our approach can be applied to design a new leader election algorithm that matches the performance of our broadcasting algorithm. Previously, all fast randomized leader election algorithms have used broadcasting as a subroutine and their complexity has been asymptotically strictly larger than the complexity of broadcasting. In particular, the fastest previously known randomized leader election algorithm of Ghaffari and Haeupler (SODA'2013) requires O(D log n/D min {log log n, log n/D} + logO(1) n)-time, succeeding with high probability. Our new algorithm again requires O(D log n/log D + logO(1) n) time, also succeeding with high probability."}],"citation":{"mla":"Czumaj, Artur, and Peter Davies. “Exploiting Spontaneous Transmissions for Broadcasting and Leader Election in Radio Networks.” <i>Journal of the ACM</i>, vol. 68, no. 2, 13, ACM, 2021, doi:<a href=\"https://doi.org/10.1145/3446383\">10.1145/3446383</a>.","ama":"Czumaj A, Davies P. Exploiting spontaneous transmissions for broadcasting and leader election in radio networks. <i>Journal of the ACM</i>. 2021;68(2). doi:<a href=\"https://doi.org/10.1145/3446383\">10.1145/3446383</a>","short":"A. Czumaj, P. Davies, Journal of the ACM 68 (2021).","ieee":"A. Czumaj and P. Davies, “Exploiting spontaneous transmissions for broadcasting and leader election in radio networks,” <i>Journal of the ACM</i>, vol. 68, no. 2. ACM, 2021.","ista":"Czumaj A, Davies P. 2021. Exploiting spontaneous transmissions for broadcasting and leader election in radio networks. Journal of the ACM. 68(2), 13.","chicago":"Czumaj, Artur, and Peter Davies. “Exploiting Spontaneous Transmissions for Broadcasting and Leader Election in Radio Networks.” <i>Journal of the ACM</i>. ACM, 2021. <a href=\"https://doi.org/10.1145/3446383\">https://doi.org/10.1145/3446383</a>.","apa":"Czumaj, A., &#38; Davies, P. (2021). Exploiting spontaneous transmissions for broadcasting and leader election in radio networks. <i>Journal of the ACM</i>. ACM. <a href=\"https://doi.org/10.1145/3446383\">https://doi.org/10.1145/3446383</a>"},"date_updated":"2026-08-12T14:03:54Z","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87"},{"main_file_link":[{"url":" https://doi.org/10.48550/arXiv.1905.11845","open_access":"1"}],"arxiv":1,"fulldoi":"https://doi.org/10.1609/aaai.v35i9.16999","article_processing_charge":"No","conference":{"location":"Virtual, Online","name":"AAAI: Conference on Artificial Intelligence","end_date":"2021-02-09","start_date":"2021-02-02"},"date_published":"2021-05-18T00:00:00Z","intvolume":"        35","publication_status":"published","quality_controlled":"1","year":"2021","doi":"10.1609/aaai.v35i9.16999","volume":35,"publisher":"AAAI Press","department":[{"_id":"DaAl"}],"month":"05","external_id":{"arxiv":["1905.11845"]},"oa":1,"day":"18","date_updated":"2026-08-19T09:25:06Z","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","page":"8209-8216","citation":{"apa":"Kungurtsev, V., Egan, M., Chatterjee, B., &#38; Alistarh, D.-A. (2021). Asynchronous optimization methods for efficient training of deep neural networks with guarantees. In <i>35th AAAI Conference on Artificial Intelligence, AAAI 2021</i> (Vol. 35, pp. 8209–8216). Virtual, Online: AAAI Press. <a href=\"https://doi.org/10.1609/aaai.v35i9.16999\">https://doi.org/10.1609/aaai.v35i9.16999</a>","chicago":"Kungurtsev, Vyacheslav, Malcolm Egan, Bapi Chatterjee, and Dan-Adrian Alistarh. “Asynchronous Optimization Methods for Efficient Training of Deep Neural Networks with Guarantees.” In <i>35th AAAI Conference on Artificial Intelligence, AAAI 2021</i>, 35:8209–16. AAAI Press, 2021. <a href=\"https://doi.org/10.1609/aaai.v35i9.16999\">https://doi.org/10.1609/aaai.v35i9.16999</a>.","ieee":"V. Kungurtsev, M. Egan, B. Chatterjee, and D.-A. Alistarh, “Asynchronous optimization methods for efficient training of deep neural networks with guarantees,” in <i>35th AAAI Conference on Artificial Intelligence, AAAI 2021</i>, Virtual, Online, 2021, vol. 35, no. 9B, pp. 8209–8216.","short":"V. Kungurtsev, M. Egan, B. Chatterjee, D.-A. Alistarh, in:, 35th AAAI Conference on Artificial Intelligence, AAAI 2021, AAAI Press, 2021, pp. 8209–8216.","ista":"Kungurtsev V, Egan M, Chatterjee B, Alistarh D-A. 2021. Asynchronous optimization methods for efficient training of deep neural networks with guarantees. 35th AAAI Conference on Artificial Intelligence, AAAI 2021. AAAI: Conference on Artificial Intelligence vol. 35, 8209–8216.","mla":"Kungurtsev, Vyacheslav, et al. “Asynchronous Optimization Methods for Efficient Training of Deep Neural Networks with Guarantees.” <i>35th AAAI Conference on Artificial Intelligence, AAAI 2021</i>, vol. 35, no. 9B, AAAI Press, 2021, pp. 8209–16, doi:<a href=\"https://doi.org/10.1609/aaai.v35i9.16999\">10.1609/aaai.v35i9.16999</a>.","ama":"Kungurtsev V, Egan M, Chatterjee B, Alistarh D-A. Asynchronous optimization methods for efficient training of deep neural networks with guarantees. In: <i>35th AAAI Conference on Artificial Intelligence, AAAI 2021</i>. Vol 35. AAAI Press; 2021:8209-8216. doi:<a href=\"https://doi.org/10.1609/aaai.v35i9.16999\">10.1609/aaai.v35i9.16999</a>"},"project":[{"name":"ISTplus - Postdoctoral Fellowships","call_identifier":"H2020","_id":"260C2330-B435-11E9-9278-68D0E5697425","grant_number":"754411"},{"grant_number":"805223","name":"Elastic Coordination for Scalable Machine Learning","call_identifier":"H2020","_id":"268A44D6-B435-11E9-9278-68D0E5697425"}],"abstract":[{"text":"Asynchronous distributed algorithms are a popular way to reduce synchronization costs in large-scale optimization, and in particular for neural network training. However, for nonsmooth and nonconvex objectives, few convergence guarantees exist beyond cases where closed-form proximal operator solutions are available. As training most popular deep neural networks corresponds to optimizing nonsmooth and nonconvex objectives, there is a pressing need for such convergence guarantees. In this paper, we analyze for the first time the convergence of stochastic asynchronous optimization for this general class of objectives. In particular, we focus on stochastic subgradient methods allowing for block variable partitioning, where the shared model is asynchronously updated by concurrent processes. To this end, we use a probabilistic model which captures key features of real asynchronous scheduling between concurrent processes. Under this model, we establish convergence with probability one to an invariant set for stochastic subgradient methods with momentum. From a practical perspective, one issue with the family of algorithms that we consider is that they are not efficiently supported by machine learning frameworks, which mostly focus on distributed data-parallel strategies. To address this, we propose a new implementation strategy for shared-memory based training of deep neural networks for a partitioned but shared model in single- and multi-GPU settings. Based on this implementation, we achieve on average1.2x speed-up in comparison to state-of-the-art training methods for popular image classification tasks, without compromising accuracy.","lang":"eng"}],"type":"conference","oa_version":"Preprint","status":"public","date_created":"2022-06-05T22:01:52Z","author":[{"last_name":"Kungurtsev","first_name":"Vyacheslav","full_name":"Kungurtsev, Vyacheslav"},{"first_name":"Malcolm","last_name":"Egan","full_name":"Egan, Malcolm"},{"last_name":"Chatterjee","first_name":"Bapi","id":"3C41A08A-F248-11E8-B48F-1D18A9856A87","full_name":"Chatterjee, Bapi","orcid":"0000-0002-2742-4028"},{"orcid":"0000-0003-3650-940X","full_name":"Alistarh, Dan-Adrian","id":"4A899BFC-F248-11E8-B48F-1D18A9856A87","last_name":"Alistarh","first_name":"Dan-Adrian"}],"title":"Asynchronous optimization methods for efficient training of deep neural networks with guarantees","_id":"11436","language":[{"iso":"eng"}],"acknowledgement":"Vyacheslav Kungurtsev was supported by the OP VVV project CZ.02.1.01/0.0/0.0/16 019/0000765 “Research Center for Informatics. Bapi Chatterjee was supported by the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No. 754411 (ISTPlus). Dan Alistarh has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 805223 ScaleML).","ec_funded":1,"publication_identifier":{"issn":["2159-5399"],"isbn":["9781713835974"],"eissn":["2374-3468"]},"scopus_import":"1","issue":"9B","publication":"35th AAAI Conference on Artificial Intelligence, AAAI 2021"},{"publisher":"Institute of Science and Technology Austria","month":"12","department":[{"_id":"GradSch"},{"_id":"DaAl"}],"day":"09","oa":1,"corr_author":"1","doi":"10.15479/at:ista:10429","file":[{"access_level":"open_access","creator":"gnadirad","relation":"main_file","success":1,"checksum":"6bf14e9a523387328f016c0689f5e10e","date_created":"2021-12-09T17:47:49Z","file_name":"Thesis_Final_09_12_2021.pdf","date_updated":"2021-12-09T17:47:49Z","content_type":"application/pdf","file_size":2370859,"file_id":"10436"},{"date_updated":"2022-03-28T12:55:12Z","content_type":"application/zip","file_name":"Thesis_Final_09_12_2021.zip","file_size":2596924,"file_id":"10437","access_level":"closed","relation":"source_file","creator":"gnadirad","date_created":"2021-12-09T17:47:49Z","checksum":"914d6c5ca86bd0add471971a8f4c4341"}],"publication_status":"published","year":"2021","ddc":["000"],"degree_awarded":"PhD","fulldoi":"https://doi.org/10.15479/at:ista:10429","has_accepted_license":"1","file_date_updated":"2022-03-28T12:55:12Z","date_published":"2021-12-09T00:00:00Z","alternative_title":["ISTA Thesis"],"article_processing_charge":"No","publication_identifier":{"issn":["2663-337X"]},"ec_funded":1,"language":[{"iso":"eng"}],"OA_place":"publisher","type":"dissertation","oa_version":"Published Version","supervisor":[{"orcid":"0000-0003-3650-940X","full_name":"Alistarh, Dan-Adrian","id":"4A899BFC-F248-11E8-B48F-1D18A9856A87","first_name":"Dan-Adrian","last_name":"Alistarh"}],"abstract":[{"text":"The scalability of concurrent data structures and distributed algorithms strongly depends on\r\nreducing the contention for shared resources and the costs of synchronization and communication. We show how such cost reductions can be attained by relaxing the strict consistency conditions required by sequential implementations. In the first part of the thesis, we consider relaxation in the context of concurrent data structures. Specifically, in data structures \r\nsuch as priority queues, imposing strong semantics renders scalability impossible, since a correct implementation of the remove operation should return only the element with highest priority. Intuitively, attempting to invoke remove operations concurrently  creates a race condition. This bottleneck  can be circumvented by relaxing semantics of the affected data structure, thus allowing removal of the elements which are no longer required to have the highest priority. We prove that the randomized implementations of relaxed data structures provide provable guarantees on the priority of the removed elements even under concurrency. Additionally, we show that in some cases the relaxed data structures can be used to scale the classical algorithms which are usually implemented with the exact ones. In the second part, we study parallel variants of the  stochastic gradient descent (SGD) algorithm, which distribute computation  among the multiple processors, thus reducing the running time. Unfortunately, in order for standard parallel SGD to succeed, each processor has to maintain a local copy of the necessary model parameter, which is identical to the local copies of other processors; the overheads from this perfect consistency in terms of communication and synchronization can negate the speedup gained by distributing the computation. We show that the consistency conditions required by SGD can be  relaxed, allowing the algorithm to be more flexible in terms of tolerating quantized communication, asynchrony, or even crash faults, while its convergence remains asymptotically the same.","lang":"eng"}],"title":"On achieving scalability through relaxation","_id":"10429","date_created":"2021-12-08T21:52:28Z","author":[{"first_name":"Giorgi","last_name":"Nadiradze","id":"3279A00C-F248-11E8-B48F-1D18A9856A87","full_name":"Nadiradze, Giorgi","orcid":"0000-0001-5634-0731"}],"status":"public","date_updated":"2026-08-19T09:30:23Z","user_id":"ba8df636-2132-11f1-aed0-ed93e2281fdd","related_material":{"record":[{"relation":"part_of_dissertation","id":"6673","status":"public"},{"relation":"part_of_dissertation","id":"5965","status":"public"},{"status":"public","id":"10435","relation":"part_of_dissertation"},{"relation":"part_of_dissertation","id":"10432","status":"public"}]},"page":"132","citation":{"chicago":"Nadiradze, Giorgi. “On Achieving Scalability through Relaxation.” Institute of Science and Technology Austria, 2021. <a href=\"https://doi.org/10.15479/at:ista:10429\">https://doi.org/10.15479/at:ista:10429</a>.","apa":"Nadiradze, G. (2021). <i>On achieving scalability through relaxation</i>. Institute of Science and Technology Austria. <a href=\"https://doi.org/10.15479/at:ista:10429\">https://doi.org/10.15479/at:ista:10429</a>","ama":"Nadiradze G. On achieving scalability through relaxation. 2021. doi:<a href=\"https://doi.org/10.15479/at:ista:10429\">10.15479/at:ista:10429</a>","mla":"Nadiradze, Giorgi. <i>On Achieving Scalability through Relaxation</i>. Institute of Science and Technology Austria, 2021, doi:<a href=\"https://doi.org/10.15479/at:ista:10429\">10.15479/at:ista:10429</a>.","ista":"Nadiradze G. 2021. On achieving scalability through relaxation. Institute of Science and Technology Austria.","ieee":"G. Nadiradze, “On achieving scalability through relaxation,” Institute of Science and Technology Austria, 2021.","short":"G. Nadiradze, On Achieving Scalability through Relaxation, Institute of Science and Technology Austria, 2021."},"project":[{"_id":"268A44D6-B435-11E9-9278-68D0E5697425","call_identifier":"H2020","name":"Elastic Coordination for Scalable Machine Learning","grant_number":"805223"}]},{"day":"18","oa":1,"external_id":{"arxiv":["2001.05918"]},"department":[{"_id":"DaAl"}],"month":"05","volume":35,"doi":"10.1609/aaai.v35i10.17092","ddc":["000"],"publication_status":"published","year":"2021","quality_controlled":"1","intvolume":"        35","conference":{"name":"AAAI: Association for the Advancement of Artificial Intelligence","start_date":"2021-02-02","end_date":"2021-02-09","location":"Virtual"},"date_published":"2021-05-18T00:00:00Z","article_processing_charge":"No","fulldoi":"https://doi.org/10.1609/aaai.v35i10.17092","main_file_link":[{"open_access":"1","url":"https://ojs.aaai.org/index.php/AAAI/article/view/17092"}],"arxiv":1,"publication":"Proceedings of the AAAI Conference on Artificial Intelligence","issue":"10","acknowledgement":"We would like to thank Christopher De Sa for his feedback on an earlier draft of this paper, as well as the anonymous AAAI reviewers for their useful comments. This project has received\r\nfunding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 805223 ScaleML). Bapi\r\nChatterjee was supported by the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No. 754411 (ISTPlus).","ec_funded":1,"language":[{"iso":"eng"}],"title":"Elastic consistency: A practical consistency model for distributed stochastic gradient descent","_id":"10432","status":"public","author":[{"full_name":"Nadiradze, Giorgi","orcid":"0000-0001-5634-0731","id":"3279A00C-F248-11E8-B48F-1D18A9856A87","last_name":"Nadiradze","first_name":"Giorgi"},{"full_name":"Markov, Ilia","first_name":"Ilia","last_name":"Markov","id":"D0CF4148-C985-11E9-8066-0BDEE5697425"},{"first_name":"Bapi","last_name":"Chatterjee","id":"3C41A08A-F248-11E8-B48F-1D18A9856A87","orcid":"0000-0002-2742-4028","full_name":"Chatterjee, Bapi"},{"full_name":"Kungurtsev, Vyacheslav ","first_name":"Vyacheslav ","last_name":"Kungurtsev"},{"full_name":"Alistarh, Dan-Adrian","orcid":"0000-0003-3650-940X","id":"4A899BFC-F248-11E8-B48F-1D18A9856A87","last_name":"Alistarh","first_name":"Dan-Adrian"}],"date_created":"2021-12-09T09:21:35Z","type":"conference","oa_version":"Published Version","abstract":[{"lang":"eng","text":"One key element behind the recent progress of machine learning has been the ability to train machine learning models in large-scale distributed shared-memory and message-passing environments. Most of these models are trained employing variants of stochastic gradient descent (SGD) based optimization, but most methods involve some type of consistency relaxation relative to sequential SGD, to mitigate its large communication or synchronization costs at scale. In this paper, we introduce a general consistency condition covering communication-reduced and asynchronous distributed SGD implementations. Our framework, called elastic consistency, decouples the system-specific aspects of the implementation from the SGD convergence requirements, giving a general way to obtain convergence bounds for a wide variety of distributed SGD methods used in practice. Elastic consistency can be used to re-derive or improve several previous convergence bounds in message-passing and shared-memory settings, but also to analyze new models and distribution schemes. As a direct application, we propose and analyze a new synchronization-avoiding scheduling scheme for distributed SGD, and show that it can be used to efficiently train deep convolutional models for image classification."}],"related_material":{"record":[{"status":"public","id":"10429","relation":"dissertation_contains"}]},"page":"9037-9045","citation":{"apa":"Nadiradze, G., Markov, I., Chatterjee, B., Kungurtsev, V., &#38; Alistarh, D.-A. (2021). Elastic consistency: A practical consistency model for distributed stochastic gradient descent. In <i>Proceedings of the AAAI Conference on Artificial Intelligence</i> (Vol. 35, pp. 9037–9045). Virtual. <a href=\"https://doi.org/10.1609/aaai.v35i10.17092\">https://doi.org/10.1609/aaai.v35i10.17092</a>","chicago":"Nadiradze, Giorgi, Ilia Markov, Bapi Chatterjee, Vyacheslav  Kungurtsev, and Dan-Adrian Alistarh. “Elastic Consistency: A Practical Consistency Model for Distributed Stochastic Gradient Descent.” In <i>Proceedings of the AAAI Conference on Artificial Intelligence</i>, 35:9037–45, 2021. <a href=\"https://doi.org/10.1609/aaai.v35i10.17092\">https://doi.org/10.1609/aaai.v35i10.17092</a>.","ista":"Nadiradze G, Markov I, Chatterjee B, Kungurtsev V, Alistarh D-A. 2021. Elastic consistency: A practical consistency model for distributed stochastic gradient descent. Proceedings of the AAAI Conference on Artificial Intelligence. AAAI: Association for the Advancement of Artificial Intelligence vol. 35, 9037–9045.","short":"G. Nadiradze, I. Markov, B. Chatterjee, V. Kungurtsev, D.-A. Alistarh, in:, Proceedings of the AAAI Conference on Artificial Intelligence, 2021, pp. 9037–9045.","ieee":"G. Nadiradze, I. Markov, B. Chatterjee, V. Kungurtsev, and D.-A. Alistarh, “Elastic consistency: A practical consistency model for distributed stochastic gradient descent,” in <i>Proceedings of the AAAI Conference on Artificial Intelligence</i>, Virtual, 2021, vol. 35, no. 10, pp. 9037–9045.","ama":"Nadiradze G, Markov I, Chatterjee B, Kungurtsev V, Alistarh D-A. Elastic consistency: A practical consistency model for distributed stochastic gradient descent. In: <i>Proceedings of the AAAI Conference on Artificial Intelligence</i>. Vol 35. ; 2021:9037-9045. doi:<a href=\"https://doi.org/10.1609/aaai.v35i10.17092\">10.1609/aaai.v35i10.17092</a>","mla":"Nadiradze, Giorgi, et al. “Elastic Consistency: A Practical Consistency Model for Distributed Stochastic Gradient Descent.” <i>Proceedings of the AAAI Conference on Artificial Intelligence</i>, vol. 35, no. 10, 2021, pp. 9037–45, doi:<a href=\"https://doi.org/10.1609/aaai.v35i10.17092\">10.1609/aaai.v35i10.17092</a>."},"project":[{"grant_number":"754411","call_identifier":"H2020","name":"ISTplus - Postdoctoral Fellowships","_id":"260C2330-B435-11E9-9278-68D0E5697425"},{"name":"Elastic Coordination for Scalable Machine Learning","call_identifier":"H2020","_id":"268A44D6-B435-11E9-9278-68D0E5697425","grant_number":"805223"}],"date_updated":"2026-08-19T09:30:23Z","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87"},{"abstract":[{"text":"While messaging systems with strong security guarantees are widely used in practice, designing a protocol that scales efficiently to large groups and enjoys similar security guarantees remains largely open. The two existing proposals to date are ART (Cohn-Gordon et al., CCS18) and TreeKEM (IETF, The Messaging Layer Security Protocol, draft). TreeKEM is the currently considered candidate by the IETF MLS working group, but dynamic group operations (i.e. adding and removing users) can cause efficiency issues. In this paper we formalize and analyze a variant of TreeKEM which we term Tainted TreeKEM (TTKEM for short). The basic idea underlying TTKEM was suggested by Millican (MLS mailing list, February 2018). This version is more efficient than TreeKEM for some natural distributions of group operations, we quantify this through simulations.Our second contribution is two security proofs for TTKEM which establish post compromise and forward secrecy even against adaptive attackers. The security loss (to the underlying PKE) in the Random Oracle Model is a polynomial factor, and a quasipolynomial one in the Standard Model. Our proofs can be adapted to TreeKEM as well. Before our work no security proof for any TreeKEM-like protocol establishing tight security against an adversary who can adaptively choose the sequence of operations was known. We also are the first to prove (or even formalize) active security where the server can arbitrarily deviate from the protocol specification. Proving fully active security – where also the users can arbitrarily deviate – remains open.","lang":"eng"}],"type":"conference","oa_version":"Preprint","date_created":"2021-09-27T13:46:27Z","author":[{"full_name":"Klein, Karen","last_name":"Klein","first_name":"Karen","id":"3E83A2F8-F248-11E8-B48F-1D18A9856A87"},{"first_name":"Guillermo","last_name":"Pascual Perez","id":"2D7ABD02-F248-11E8-B48F-1D18A9856A87","orcid":"0000-0001-8630-415X","full_name":"Pascual Perez, Guillermo"},{"id":"488F98B0-F248-11E8-B48F-1D18A9856A87","last_name":"Walter","first_name":"Michael","orcid":"0000-0003-3186-2482","full_name":"Walter, Michael"},{"id":"4BD3F30E-F248-11E8-B48F-1D18A9856A87","last_name":"Kamath Hosdurg","first_name":"Chethan","full_name":"Kamath Hosdurg, Chethan","orcid":"0009-0006-6812-7317"},{"full_name":"Capretto, Margarita","last_name":"Capretto","first_name":"Margarita"},{"full_name":"Cueto Noval, Miguel","orcid":"0000-0002-2505-4246","id":"ffc563a3-f6e0-11ea-865d-e3cce03d17cc","first_name":"Miguel","last_name":"Cueto Noval"},{"full_name":"Markov, Ilia","last_name":"Markov","first_name":"Ilia","id":"D0CF4148-C985-11E9-8066-0BDEE5697425"},{"full_name":"Yeo, Michelle X","orcid":"0009-0001-3676-4809","id":"2D82B818-F248-11E8-B48F-1D18A9856A87","last_name":"Yeo","first_name":"Michelle X"},{"id":"2A8DFA8C-F248-11E8-B48F-1D18A9856A87","first_name":"Joel F","last_name":"Alwen","full_name":"Alwen, Joel F"},{"full_name":"Pietrzak, Krzysztof Z","orcid":"0000-0002-9139-1654","id":"3E04A7AA-F248-11E8-B48F-1D18A9856A87","first_name":"Krzysztof Z","last_name":"Pietrzak"}],"status":"public","title":"Keep the dirt: tainted TreeKEM, adaptively and actively secure continuous group key agreement","_id":"10049","date_updated":"2026-10-16T08:11:07Z","user_id":"317138e5-6ab7-11ef-aa6d-ffef3953e345","page":"268-284","citation":{"apa":"Klein, K., Pascual Perez, G., Walter, M., Kamath Hosdurg, C., Capretto, M., Cueto Noval, M., … Pietrzak, K. Z. (2021). Keep the dirt: tainted TreeKEM, adaptively and actively secure continuous group key agreement. In <i>2021 IEEE Symposium on Security and Privacy </i> (pp. 268–284). San Francisco, CA, United States: IEEE. <a href=\"https://doi.org/10.1109/sp40001.2021.00035\">https://doi.org/10.1109/sp40001.2021.00035</a>","chicago":"Klein, Karen, Guillermo Pascual Perez, Michael Walter, Chethan Kamath Hosdurg, Margarita Capretto, Miguel Cueto Noval, Ilia Markov, Michelle X Yeo, Joel F Alwen, and Krzysztof Z Pietrzak. “Keep the Dirt: Tainted TreeKEM, Adaptively and Actively Secure Continuous Group Key Agreement.” In <i>2021 IEEE Symposium on Security and Privacy </i>, 268–84. IEEE, 2021. <a href=\"https://doi.org/10.1109/sp40001.2021.00035\">https://doi.org/10.1109/sp40001.2021.00035</a>.","ieee":"K. Klein <i>et al.</i>, “Keep the dirt: tainted TreeKEM, adaptively and actively secure continuous group key agreement,” in <i>2021 IEEE Symposium on Security and Privacy </i>, San Francisco, CA, United States, 2021, pp. 268–284.","short":"K. Klein, G. Pascual Perez, M. Walter, C. Kamath Hosdurg, M. Capretto, M. Cueto Noval, I. Markov, M.X. Yeo, J.F. Alwen, K.Z. Pietrzak, in:, 2021 IEEE Symposium on Security and Privacy , IEEE, 2021, pp. 268–284.","ista":"Klein K, Pascual Perez G, Walter M, Kamath Hosdurg C, Capretto M, Cueto Noval M, Markov I, Yeo MX, Alwen JF, Pietrzak KZ. 2021. Keep the dirt: tainted TreeKEM, adaptively and actively secure continuous group key agreement. 2021 IEEE Symposium on Security and Privacy . SP: Symposium on Security and Privacy, 268–284.","mla":"Klein, Karen, et al. “Keep the Dirt: Tainted TreeKEM, Adaptively and Actively Secure Continuous Group Key Agreement.” <i>2021 IEEE Symposium on Security and Privacy </i>, IEEE, 2021, pp. 268–84, doi:<a href=\"https://doi.org/10.1109/sp40001.2021.00035\">10.1109/sp40001.2021.00035</a>.","ama":"Klein K, Pascual Perez G, Walter M, et al. Keep the dirt: tainted TreeKEM, adaptively and actively secure continuous group key agreement. In: <i>2021 IEEE Symposium on Security and Privacy </i>. IEEE; 2021:268-284. doi:<a href=\"https://doi.org/10.1109/sp40001.2021.00035\">10.1109/sp40001.2021.00035</a>"},"project":[{"_id":"2564DBCA-B435-11E9-9278-68D0E5697425","call_identifier":"H2020","name":"International IST Doctoral Program","grant_number":"665385"},{"_id":"258AA5B2-B435-11E9-9278-68D0E5697425","call_identifier":"H2020","name":"Teaching Old Crypto New Tricks","grant_number":"682815"}],"related_material":{"record":[{"status":"public","relation":"dissertation_contains","id":"10035"},{"id":"18088","relation":"dissertation_contains","status":"public"}]},"acknowledgement":"The first three authors contributed equally to this work. Funded by the European Research Council (ERC) under the European Union’s Horizon2020 research and innovation programme (682815-TOCNeT). Funded by the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie Grant Agreement No.665385.","ec_funded":1,"scopus_import":"1","publication":"2021 IEEE Symposium on Security and Privacy ","language":[{"iso":"eng"}],"publication_status":"published","year":"2021","quality_controlled":"1","main_file_link":[{"url":"https://eprint.iacr.org/2019/1489","open_access":"1"}],"fulldoi":"https://doi.org/10.1109/sp40001.2021.00035","article_processing_charge":"No","conference":{"start_date":"2021-05-24","end_date":"2021-05-27","name":"SP: Symposium on Security and Privacy","location":"San Francisco, CA, United States"},"date_published":"2021-08-26T00:00:00Z","publisher":"IEEE","month":"08","department":[{"_id":"KrPi"},{"_id":"DaAl"}],"external_id":{"isi":["001316065000016"]},"corr_author":"1","oa":1,"day":"26","doi":"10.1109/sp40001.2021.00035","isi":1},{"oa_version":"Published Version","type":"conference","abstract":[{"text":"We consider the following dynamic load-balancing process: given an underlying graph G with n nodes, in each step t≥ 0, one unit of load is created, and placed at a randomly chosen graph node. In the same step, the chosen node picks a random neighbor, and the two nodes balance their loads by averaging them. We are interested in the expected gap between the minimum and maximum loads at nodes as the process progresses, and its dependence on n and on the graph structure. Variants of the above graphical balanced allocation process have been studied previously by Peres, Talwar, and Wieder [Peres et al., 2015], and by Sauerwald and Sun [Sauerwald and Sun, 2015]. These authors left as open the question of characterizing the gap in the case of cycle graphs in the dynamic case, where weights are created during the algorithm’s execution. For this case, the only known upper bound is of 𝒪(n log n), following from a majorization argument due to [Peres et al., 2015], which analyzes a related graphical allocation process. In this paper, we provide an upper bound of 𝒪 (√n log n) on the expected gap of the above process for cycles of length n. We introduce a new potential analysis technique, which enables us to bound the difference in load between k-hop neighbors on the cycle, for any k ≤ n/2. We complement this with a \"gap covering\" argument, which bounds the maximum value of the gap by bounding its value across all possible subsets of a certain structure, and recursively bounding the gaps within each subset. We provide analytical and experimental evidence that our upper bound on the gap is tight up to a logarithmic factor.","lang":"eng"}],"_id":"15077","title":"Dynamic averaging load balancing on cycles","status":"public","date_created":"2024-03-05T07:25:37Z","author":[{"orcid":"0000-0003-3650-940X","full_name":"Alistarh, Dan-Adrian","id":"4A899BFC-F248-11E8-B48F-1D18A9856A87","last_name":"Alistarh","first_name":"Dan-Adrian"},{"id":"3279A00C-F248-11E8-B48F-1D18A9856A87","last_name":"Nadiradze","first_name":"Giorgi","full_name":"Nadiradze, Giorgi","orcid":"0000-0001-5634-0731"},{"full_name":"Sabour, Amirmojtaba","id":"bcc145fd-e77f-11ea-ae8b-80d661dbff67","last_name":"Sabour","first_name":"Amirmojtaba"}],"user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","date_updated":"2025-07-10T11:55:11Z","related_material":{"record":[{"relation":"later_version","id":"8286","status":"public"}]},"license":"https://creativecommons.org/licenses/by/3.0/","project":[{"grant_number":"805223","call_identifier":"H2020","name":"Elastic Coordination for Scalable Machine Learning","_id":"268A44D6-B435-11E9-9278-68D0E5697425"}],"citation":{"apa":"Alistarh, D.-A., Nadiradze, G., &#38; Sabour, A. (2020). Dynamic averaging load balancing on cycles. In <i>47th International Colloquium on Automata, Languages, and Programming</i> (Vol. 168). Saarbrücken, Germany, Virtual: Schloss Dagstuhl - Leibniz-Zentrum für Informatik. <a href=\"https://doi.org/10.4230/LIPIcs.ICALP.2020.7\">https://doi.org/10.4230/LIPIcs.ICALP.2020.7</a>","chicago":"Alistarh, Dan-Adrian, Giorgi Nadiradze, and Amirmojtaba Sabour. “Dynamic Averaging Load Balancing on Cycles.” In <i>47th International Colloquium on Automata, Languages, and Programming</i>, Vol. 168. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2020. <a href=\"https://doi.org/10.4230/LIPIcs.ICALP.2020.7\">https://doi.org/10.4230/LIPIcs.ICALP.2020.7</a>.","short":"D.-A. Alistarh, G. Nadiradze, A. Sabour, in:, 47th International Colloquium on Automata, Languages, and Programming, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2020.","ista":"Alistarh D-A, Nadiradze G, Sabour A. 2020. Dynamic averaging load balancing on cycles. 47th International Colloquium on Automata, Languages, and Programming. ICALP: Automata, Languages and Programming, LIPIcs, vol. 168, 7.","ieee":"D.-A. Alistarh, G. Nadiradze, and A. Sabour, “Dynamic averaging load balancing on cycles,” in <i>47th International Colloquium on Automata, Languages, and Programming</i>, Saarbrücken, Germany, Virtual, 2020, vol. 168.","mla":"Alistarh, Dan-Adrian, et al. “Dynamic Averaging Load Balancing on Cycles.” <i>47th International Colloquium on Automata, Languages, and Programming</i>, vol. 168, 7, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2020, doi:<a href=\"https://doi.org/10.4230/LIPIcs.ICALP.2020.7\">10.4230/LIPIcs.ICALP.2020.7</a>.","ama":"Alistarh D-A, Nadiradze G, Sabour A. Dynamic averaging load balancing on cycles. In: <i>47th International Colloquium on Automata, Languages, and Programming</i>. Vol 168. Schloss Dagstuhl - Leibniz-Zentrum für Informatik; 2020. doi:<a href=\"https://doi.org/10.4230/LIPIcs.ICALP.2020.7\">10.4230/LIPIcs.ICALP.2020.7</a>"},"acknowledgement":"The authors sincerely thank Thomas Sauerwald and George Giakkoupis for insightful discussions, and Mohsen Ghaffari, Yuval Peres, and Udi Wieder for feedback on earlier\r\nversions of this draft. We also thank the ICALP anonymous reviewers for their very useful comments.\r\nFunding: European Research Council funding award PR1042ERC01","ec_funded":1,"publication":"47th International Colloquium on Automata, Languages, and Programming","scopus_import":"1","language":[{"iso":"eng"}],"year":"2020","quality_controlled":"1","publication_status":"published","intvolume":"       168","tmp":{"image":"/images/cc_by.png","short":"CC BY (3.0)","name":"Creative Commons Attribution 3.0 Unported (CC BY 3.0)","legal_code_url":"https://creativecommons.org/licenses/by/3.0/legalcode"},"ddc":["000"],"fulldoi":"https://doi.org/10.4230/LIPIcs.ICALP.2020.7","arxiv":1,"date_published":"2020-06-29T00:00:00Z","file_date_updated":"2024-03-05T07:25:15Z","alternative_title":["LIPIcs"],"conference":{"location":"Saarbrücken, Germany, Virtual","name":"ICALP: Automata, Languages and Programming","end_date":"2020-07-11","start_date":"2020-07-08"},"has_accepted_license":"1","article_processing_charge":"No","external_id":{"arxiv":["2003.09297"]},"month":"06","department":[{"_id":"DaAl"}],"publisher":"Schloss Dagstuhl - Leibniz-Zentrum für Informatik","day":"29","corr_author":"1","oa":1,"volume":168,"file":[{"file_id":"15078","file_size":782987,"file_name":"2020_LIPIcs_Alistarh.pdf","date_updated":"2024-03-05T07:25:15Z","content_type":"application/pdf","date_created":"2024-03-05T07:25:15Z","checksum":"e5eb16199f4ccfd77a321977eb3f026f","success":1,"creator":"dernst","relation":"main_file","access_level":"open_access"}],"doi":"10.4230/LIPIcs.ICALP.2020.7","article_number":"7"},{"day":"10","oa":1,"external_id":{"arxiv":["2010.12460"]},"publisher":"Neural Information Processing Systems Foundation","month":"12","department":[{"_id":"DaAl"}],"volume":33,"publication_status":"published","quality_controlled":"1","year":"2020","intvolume":"        33","conference":{"location":"Vancouver, Canada","name":"NeurIPS: Neural Information Processing Systems","end_date":"2020-12-12","start_date":"2020-12-06"},"date_published":"2020-12-10T00:00:00Z","alternative_title":["NeurIPS"],"article_processing_charge":"No","main_file_link":[{"url":"https://doi.org/10.48550/arXiv.2010.12460","open_access":"1"}],"arxiv":1,"publication":"Advances in Neural Information Processing Systems","ec_funded":1,"publication_identifier":{"isbn":["9781713829546"]},"acknowledgement":"The authors would like to thank Blair Bilodeau, David Fleet, Mufan Li, and Jeffrey Negrea for\r\nhelpful discussions. FF was supported by OGS Scholarship. DA and IM were supported the\r\nEuropean Research Council (ERC) under the European Union’s Horizon 2020 research and innovation\r\nprogramme (grant agreement No 805223 ScaleML). DMR was supported by an NSERC Discovery\r\nGrant. ARK was supported by NSERC Postdoctoral Fellowship. Resources used in preparing this research were provided, in part, by the Province of Ontario, the Government of Canada through CIFAR, and companies sponsoring the Vector Institute.","language":[{"iso":"eng"}],"title":"Adaptive gradient quantization for data-parallel SGD","_id":"15086","date_created":"2024-03-06T08:35:58Z","author":[{"last_name":"Faghri","first_name":"Fartash ","full_name":"Faghri, Fartash "},{"full_name":"Tabrizian, Iman ","last_name":"Tabrizian","first_name":"Iman "},{"first_name":"Ilia","last_name":"Markov","id":"D0CF4148-C985-11E9-8066-0BDEE5697425","full_name":"Markov, Ilia"},{"last_name":"Alistarh","first_name":"Dan-Adrian","id":"4A899BFC-F248-11E8-B48F-1D18A9856A87","orcid":"0000-0003-3650-940X","full_name":"Alistarh, Dan-Adrian"},{"full_name":"Roy, Daniel ","last_name":"Roy","first_name":"Daniel "},{"full_name":"Ramezani-Kebrya, Ali ","last_name":"Ramezani-Kebrya","first_name":"Ali "}],"status":"public","type":"conference","oa_version":"Preprint","abstract":[{"text":"Many communication-efficient variants of SGD use gradient quantization schemes. These schemes are often heuristic and fixed over the course of training. We empirically observe that the statistics of gradients of deep models change during the training. Motivated by this observation, we introduce two adaptive quantization schemes, ALQ and AMQ. In both schemes, processors update their compression schemes in parallel by efficiently computing sufficient statistics of a parametric distribution. We improve the validation accuracy by almost 2% on CIFAR-10 and 1% on ImageNet in challenging low-cost communication setups. Our adaptive methods are also significantly more robust to the choice of hyperparameters.\r\n\r\n","lang":"eng"}],"citation":{"chicago":"Faghri, Fartash , Iman  Tabrizian, Ilia Markov, Dan-Adrian Alistarh, Daniel  Roy, and Ali  Ramezani-Kebrya. “Adaptive Gradient Quantization for Data-Parallel SGD.” In <i>Advances in Neural Information Processing Systems</i>, Vol. 33. Neural Information Processing Systems Foundation, 2020.","apa":"Faghri, F., Tabrizian, I., Markov, I., Alistarh, D.-A., Roy, D., &#38; Ramezani-Kebrya, A. (2020). Adaptive gradient quantization for data-parallel SGD. In <i>Advances in Neural Information Processing Systems</i> (Vol. 33). Vancouver, Canada: Neural Information Processing Systems Foundation.","mla":"Faghri, Fartash, et al. “Adaptive Gradient Quantization for Data-Parallel SGD.” <i>Advances in Neural Information Processing Systems</i>, vol. 33, Neural Information Processing Systems Foundation, 2020.","ama":"Faghri F, Tabrizian I, Markov I, Alistarh D-A, Roy D, Ramezani-Kebrya A. Adaptive gradient quantization for data-parallel SGD. In: <i>Advances in Neural Information Processing Systems</i>. Vol 33. Neural Information Processing Systems Foundation; 2020.","ista":"Faghri F, Tabrizian I, Markov I, Alistarh D-A, Roy D, Ramezani-Kebrya A. 2020. Adaptive gradient quantization for data-parallel SGD. Advances in Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems, NeurIPS, vol. 33.","short":"F. Faghri, I. Tabrizian, I. Markov, D.-A. Alistarh, D. Roy, A. Ramezani-Kebrya, in:, Advances in Neural Information Processing Systems, Neural Information Processing Systems Foundation, 2020.","ieee":"F. Faghri, I. Tabrizian, I. Markov, D.-A. Alistarh, D. Roy, and A. Ramezani-Kebrya, “Adaptive gradient quantization for data-parallel SGD,” in <i>Advances in Neural Information Processing Systems</i>, Vancouver, Canada, 2020, vol. 33."},"project":[{"_id":"268A44D6-B435-11E9-9278-68D0E5697425","name":"Elastic Coordination for Scalable Machine Learning","call_identifier":"H2020","grant_number":"805223"}],"date_updated":"2025-04-14T07:49:16Z","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87"},{"oa_version":"Published Version","type":"conference","abstract":[{"lang":"eng","text":"Union-Find (or Disjoint-Set Union) is one of the fundamental problems in computer science; it has been well-studied from both theoretical and practical perspectives in the sequential case. Recently, there has been mounting interest in analyzing this problem in the concurrent scenario, and several asymptotically-efficient algorithms have been proposed. Yet, to date, there is very little known about the practical performance of concurrent Union-Find. This work addresses this gap. We evaluate and analyze the performance of several concurrent Union-Find algorithms and optimization strategies across a wide range of platforms (Intel, AMD, and ARM) and workloads (social, random, and road networks, as well as integrations into more complex algorithms). We first observe that, due to the limited computational cost, the number of induced cache misses is the critical determining factor for the performance of existing algorithms. We introduce new techniques to reduce this cost by storing node priorities implicitly and by using plain reads and writes in a way that does not affect the correctness of the algorithms. Finally, we show that Union-Find implementations are an interesting application for Transactional Memory (TM): one of the fastest algorithm variants we discovered is a sequential one that uses coarse-grained locking with the lock elision optimization to reduce synchronization cost and increase scalability. "}],"title":"In search of the fastest concurrent union-find algorithm","_id":"7605","date_created":"2020-03-22T23:00:46Z","author":[{"id":"4A899BFC-F248-11E8-B48F-1D18A9856A87","first_name":"Dan-Adrian","last_name":"Alistarh","full_name":"Alistarh, Dan-Adrian","orcid":"0000-0003-3650-940X"},{"first_name":"Alexander","last_name":"Fedorov","full_name":"Fedorov, Alexander"},{"id":"2F4DB10C-F248-11E8-B48F-1D18A9856A87","first_name":"Nikita","last_name":"Koval","full_name":"Koval, Nikita"}],"status":"public","date_updated":"2025-07-10T11:54:46Z","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","citation":{"chicago":"Alistarh, Dan-Adrian, Alexander Fedorov, and Nikita Koval. “In Search of the Fastest Concurrent Union-Find Algorithm.” In <i>23rd International Conference on Principles of Distributed Systems</i>, 153:15:1-15:16. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2020. <a href=\"https://doi.org/10.4230/LIPIcs.OPODIS.2019.15\">https://doi.org/10.4230/LIPIcs.OPODIS.2019.15</a>.","apa":"Alistarh, D.-A., Fedorov, A., &#38; Koval, N. (2020). In search of the fastest concurrent union-find algorithm. In <i>23rd International Conference on Principles of Distributed Systems</i> (Vol. 153, p. 15:1-15:16). Neuchatal, Switzerland: Schloss Dagstuhl - Leibniz-Zentrum für Informatik. <a href=\"https://doi.org/10.4230/LIPIcs.OPODIS.2019.15\">https://doi.org/10.4230/LIPIcs.OPODIS.2019.15</a>","mla":"Alistarh, Dan-Adrian, et al. “In Search of the Fastest Concurrent Union-Find Algorithm.” <i>23rd International Conference on Principles of Distributed Systems</i>, vol. 153, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2020, p. 15:1-15:16, doi:<a href=\"https://doi.org/10.4230/LIPIcs.OPODIS.2019.15\">10.4230/LIPIcs.OPODIS.2019.15</a>.","ama":"Alistarh D-A, Fedorov A, Koval N. In search of the fastest concurrent union-find algorithm. In: <i>23rd International Conference on Principles of Distributed Systems</i>. Vol 153. Schloss Dagstuhl - Leibniz-Zentrum für Informatik; 2020:15:1-15:16. doi:<a href=\"https://doi.org/10.4230/LIPIcs.OPODIS.2019.15\">10.4230/LIPIcs.OPODIS.2019.15</a>","short":"D.-A. Alistarh, A. Fedorov, N. Koval, in:, 23rd International Conference on Principles of Distributed Systems, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2020, p. 15:1-15:16.","ista":"Alistarh D-A, Fedorov A, Koval N. 2020. In search of the fastest concurrent union-find algorithm. 23rd International Conference on Principles of Distributed Systems. OPODIS: International Conference on Principles of Distributed Systems, LIPIcs, vol. 153, 15:1-15:16.","ieee":"D.-A. Alistarh, A. Fedorov, and N. Koval, “In search of the fastest concurrent union-find algorithm,” in <i>23rd International Conference on Principles of Distributed Systems</i>, Neuchatal, Switzerland, 2020, vol. 153, p. 15:1-15:16."},"page":"15:1-15:16","publication_identifier":{"issn":["1868-8969"],"isbn":["9783959771337"]},"publication":"23rd International Conference on Principles of Distributed Systems","scopus_import":"1","language":[{"iso":"eng"}],"publication_status":"published","quality_controlled":"1","year":"2020","intvolume":"       153","ddc":["000"],"tmp":{"image":"/images/cc_by.png","short":"CC BY (3.0)","name":"Creative Commons Attribution 3.0 Unported (CC BY 3.0)","legal_code_url":"https://creativecommons.org/licenses/by/3.0/legalcode"},"fulldoi":"https://doi.org/10.4230/LIPIcs.OPODIS.2019.15","arxiv":1,"conference":{"name":"OPODIS: International Conference on Principles of Distributed Systems","end_date":"2019-12-19","start_date":"2019-12-17","location":"Neuchatal, Switzerland"},"has_accepted_license":"1","file_date_updated":"2020-07-14T12:48:01Z","alternative_title":["LIPIcs"],"date_published":"2020-02-01T00:00:00Z","article_processing_charge":"No","external_id":{"arxiv":["1911.06347"]},"publisher":"Schloss Dagstuhl - Leibniz-Zentrum für Informatik","month":"02","department":[{"_id":"DaAl"}],"day":"01","oa":1,"corr_author":"1","volume":153,"doi":"10.4230/LIPIcs.OPODIS.2019.15","file":[{"date_created":"2020-03-23T09:22:48Z","checksum":"d66f07ecb609d9f02433e39f80a447e9","creator":"dernst","relation":"main_file","access_level":"open_access","file_id":"7609","file_size":13074131,"date_updated":"2020-07-14T12:48:01Z","content_type":"application/pdf","file_name":"2019_LIPIcs_Alistarh.pdf"}]},{"author":[{"full_name":"Koval, Nikita","first_name":"Nikita","last_name":"Koval","id":"2F4DB10C-F248-11E8-B48F-1D18A9856A87"},{"first_name":"Mariia","last_name":"Sokolova","id":"26217AE4-77FF-11EA-8101-AD24D49E41F4","full_name":"Sokolova, Mariia"},{"last_name":"Fedorov","first_name":"Alexander","full_name":"Fedorov, Alexander"},{"id":"4A899BFC-F248-11E8-B48F-1D18A9856A87","first_name":"Dan-Adrian","last_name":"Alistarh","full_name":"Alistarh, Dan-Adrian","orcid":"0000-0003-3650-940X"},{"full_name":"Tsitelov, Dmitry","last_name":"Tsitelov","first_name":"Dmitry"}],"status":"public","date_created":"2020-04-05T22:00:48Z","title":"Testing concurrency on the JVM with Lincheck","_id":"7635","abstract":[{"lang":"eng","text":"Concurrent programming can be notoriously complex and error-prone. Programming bugs can arise from a variety of sources, such as operation re-reordering, or incomplete understanding of the memory model. A variety of formal and model checking methods have been developed to address this fundamental difficulty. While technically interesting, existing academic methods are still hard to apply to the large codebases typical of industrial deployments, which limits their practical impact."}],"type":"conference","oa_version":"Published Version","citation":{"ama":"Koval N, Sokolova M, Fedorov A, Alistarh D-A, Tsitelov D. Testing concurrency on the JVM with Lincheck. In: <i>Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming</i>. Association for Computing Machinery; 2020:423-424. doi:<a href=\"https://doi.org/10.1145/3332466.3374503\">10.1145/3332466.3374503</a>","mla":"Koval, Nikita, et al. “Testing Concurrency on the JVM with Lincheck.” <i>Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming</i>, Association for Computing Machinery, 2020, pp. 423–24, doi:<a href=\"https://doi.org/10.1145/3332466.3374503\">10.1145/3332466.3374503</a>.","ieee":"N. Koval, M. Sokolova, A. Fedorov, D.-A. Alistarh, and D. Tsitelov, “Testing concurrency on the JVM with Lincheck,” in <i>Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming</i>, San Diego, CA, United States, 2020, pp. 423–424.","short":"N. Koval, M. Sokolova, A. Fedorov, D.-A. Alistarh, D. Tsitelov, in:, Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, Association for Computing Machinery, 2020, pp. 423–424.","ista":"Koval N, Sokolova M, Fedorov A, Alistarh D-A, Tsitelov D. 2020. Testing concurrency on the JVM with Lincheck. Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. PPOPP: Principles and Practice of Parallel Programming, 423–424.","chicago":"Koval, Nikita, Mariia Sokolova, Alexander Fedorov, Dan-Adrian Alistarh, and Dmitry Tsitelov. “Testing Concurrency on the JVM with Lincheck.” In <i>Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming</i>, 423–24. Association for Computing Machinery, 2020. <a href=\"https://doi.org/10.1145/3332466.3374503\">https://doi.org/10.1145/3332466.3374503</a>.","apa":"Koval, N., Sokolova, M., Fedorov, A., Alistarh, D.-A., &#38; Tsitelov, D. (2020). Testing concurrency on the JVM with Lincheck. In <i>Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming</i> (pp. 423–424). San Diego, CA, United States: Association for Computing Machinery. <a href=\"https://doi.org/10.1145/3332466.3374503\">https://doi.org/10.1145/3332466.3374503</a>"},"page":"423-424","date_updated":"2026-06-18T19:24:47Z","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","scopus_import":"1","publication":"Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming","publication_identifier":{"isbn":["9781450368186"]},"OA_type":"free access","OA_place":"publisher","language":[{"iso":"eng"}],"ddc":["000"],"publication_status":"published","year":"2020","quality_controlled":"1","article_processing_charge":"No","conference":{"location":"San Diego, CA, United States","start_date":"2020-02-22","end_date":"2020-02-26","name":"PPOPP: Principles and Practice of Parallel Programming"},"date_published":"2020-02-19T00:00:00Z","main_file_link":[{"open_access":"1","url":"https://doi.org/10.1145/3332466.3374503"}],"fulldoi":"https://doi.org/10.1145/3332466.3374503","oa":1,"day":"19","publisher":"Association for Computing Machinery","month":"02","department":[{"_id":"DaAl"}],"doi":"10.1145/3332466.3374503"},{"related_material":{"record":[{"id":"9541","relation":"later_version","status":"public"}]},"page":"175-185","citation":{"apa":"Czumaj, A., Davies, P., &#38; Parter, M. (2020). Graph sparsification for derandomizing massively parallel computation with low space. In <i>Proceedings of the 32nd ACM Symposium on Parallelism in Algorithms and Architectures (SPAA 2020)</i> (pp. 175–185). Virtual Event, United States: Association for Computing Machinery. <a href=\"https://doi.org/10.1145/3350755.3400282\">https://doi.org/10.1145/3350755.3400282</a>","chicago":"Czumaj, Artur, Peter Davies, and Merav Parter. “Graph Sparsification for Derandomizing Massively Parallel Computation with Low Space.” In <i>Proceedings of the 32nd ACM Symposium on Parallelism in Algorithms and Architectures (SPAA 2020)</i>, 175–85. Association for Computing Machinery, 2020. <a href=\"https://doi.org/10.1145/3350755.3400282\">https://doi.org/10.1145/3350755.3400282</a>.","ista":"Czumaj A, Davies P, Parter M. 2020. Graph sparsification for derandomizing massively parallel computation with low space. Proceedings of the 32nd ACM Symposium on Parallelism in Algorithms and Architectures (SPAA 2020). SPAA: Symposium on Parallelism in Algorithms and Architectures, 175–185.","short":"A. Czumaj, P. Davies, M. Parter, in:, Proceedings of the 32nd ACM Symposium on Parallelism in Algorithms and Architectures (SPAA 2020), Association for Computing Machinery, 2020, pp. 175–185.","ieee":"A. Czumaj, P. Davies, and M. Parter, “Graph sparsification for derandomizing massively parallel computation with low space,” in <i>Proceedings of the 32nd ACM Symposium on Parallelism in Algorithms and Architectures (SPAA 2020)</i>, Virtual Event, United States, 2020, no. 7, pp. 175–185.","ama":"Czumaj A, Davies P, Parter M. Graph sparsification for derandomizing massively parallel computation with low space. In: <i>Proceedings of the 32nd ACM Symposium on Parallelism in Algorithms and Architectures (SPAA 2020)</i>. Association for Computing Machinery; 2020:175-185. doi:<a href=\"https://doi.org/10.1145/3350755.3400282\">10.1145/3350755.3400282</a>","mla":"Czumaj, Artur, et al. “Graph Sparsification for Derandomizing Massively Parallel Computation with Low Space.” <i>Proceedings of the 32nd ACM Symposium on Parallelism in Algorithms and Architectures (SPAA 2020)</i>, no. 7, Association for Computing Machinery, 2020, pp. 175–85, doi:<a href=\"https://doi.org/10.1145/3350755.3400282\">10.1145/3350755.3400282</a>."},"project":[{"call_identifier":"H2020","name":"ISTplus - Postdoctoral Fellowships","_id":"260C2330-B435-11E9-9278-68D0E5697425","grant_number":"754411"}],"date_updated":"2025-04-15T06:54:47Z","user_id":"4359f0d1-fa6c-11eb-b949-802e58b17ae8","title":"Graph sparsification for derandomizing massively parallel computation with low space","_id":"7802","status":"public","date_created":"2020-05-06T08:53:34Z","author":[{"orcid":"0000-0002-5646-9524","full_name":"Czumaj, Artur","last_name":"Czumaj","first_name":"Artur"},{"orcid":"0000-0002-5646-9524","full_name":"Davies, Peter","id":"11396234-BB50-11E9-B24C-90FCE5697425","last_name":"Davies","first_name":"Peter"},{"first_name":"Merav","last_name":"Parter","full_name":"Parter, Merav"}],"type":"conference","oa_version":"Preprint","abstract":[{"text":"The Massively Parallel Computation (MPC) model is an emerging model which distills core  aspects of distributed and parallel computation. It has been developed as a tool to solve (typically graph) problems in systems where the input is distributed over many machines with limited space.\r\n\t\r\nRecent work has focused on the regime in which machines have sublinear (in $n$, the number of nodes in the input graph) space, with randomized algorithms presented for fundamental graph problems of Maximal Matching and Maximal Independent Set. However, there have been no prior corresponding deterministic algorithms.\r\n\t\r\n\tA major challenge underlying the sublinear space setting is that the local space of each machine might be too small to store all the edges incident to a single node. This poses a considerable obstacle compared to the classical models in which each node is assumed to know and have easy access to its incident edges. To overcome this barrier we introduce a new graph sparsification technique that deterministically computes a low-degree subgraph with additional desired properties. The degree of the nodes in this subgraph is small in the sense that the edges of each node can be now stored on a single machine. This low-degree subgraph also has the property that solving the problem on this subgraph provides \\emph{significant} global progress, i.e., progress towards solving the problem for the original input graph.\r\n\t\r\nUsing this framework to derandomize the well-known randomized algorithm of Luby [SICOMP'86], we obtain $O(\\log \\Delta+\\log\\log n)$-round deterministic MPC algorithms for solving the fundamental problems of Maximal Matching and Maximal Independent Set with $O(n^{\\epsilon})$ space on each machine for any constant $\\epsilon > 0$. Based on the recent work of Ghaffari et al. [FOCS'18], this additive $O(\\log\\log n)$ factor is conditionally essential. These algorithms can also be shown to run in $O(\\log \\Delta)$ rounds in the closely related model of CONGESTED CLIQUE, improving upon the state-of-the-art bound of $O(\\log^2 \\Delta)$ rounds by Censor-Hillel et al. [DISC'17].","lang":"eng"}],"language":[{"iso":"eng"}],"publication":"Proceedings of the 32nd ACM Symposium on Parallelism in Algorithms and Architectures (SPAA 2020)","scopus_import":"1","issue":"7","ec_funded":1,"conference":{"location":"Virtual Event, United States","end_date":"2020-07-17","start_date":"2020-07-15","name":"SPAA: Symposium on Parallelism in Algorithms and Architectures"},"date_published":"2020-07-01T00:00:00Z","article_processing_charge":"No","fulldoi":"https://doi.org/10.1145/3350755.3400282","main_file_link":[{"url":"https://arxiv.org/abs/1912.05390","open_access":"1"}],"arxiv":1,"publication_status":"published","quality_controlled":"1","year":"2020","isi":1,"doi":"10.1145/3350755.3400282","day":"01","oa":1,"external_id":{"isi":["000744436200015"],"arxiv":["1912.05390"]},"publisher":"Association for Computing Machinery","month":"07","department":[{"_id":"DaAl"}]}]