--- _id: '6972' abstract: - lang: eng text: 'We give fault-tolerant algorithms for establishing synchrony in distributed systems in which each of thennodes has its own clock. Our algorithms operate in a very strong fault model: we require self-stabilisation, i.e.,the initial state of the system may be arbitrary, and there can be up to fJournal of the ACM. 2019;66(5). doi:10.1145/3339471 apa: Lenzen, C., & Rybicki, J. (2019). Self-stabilising Byzantine clock synchronisation is almost as easy as consensus. Journal of the ACM. ACM. https://doi.org/10.1145/3339471 chicago: Lenzen, Christoph, and Joel Rybicki. “Self-Stabilising Byzantine Clock Synchronisation Is Almost as Easy as Consensus.” Journal of the ACM. ACM, 2019. https://doi.org/10.1145/3339471. ieee: C. Lenzen and J. Rybicki, “Self-stabilising Byzantine clock synchronisation is almost as easy as consensus,” Journal of the ACM, vol. 66, no. 5. ACM, 2019. ista: Lenzen C, Rybicki J. 2019. Self-stabilising Byzantine clock synchronisation is almost as easy as consensus. Journal of the ACM. 66(5), 32. mla: Lenzen, Christoph, and Joel Rybicki. “Self-Stabilising Byzantine Clock Synchronisation Is Almost as Easy as Consensus.” Journal of the ACM, vol. 66, no. 5, 32, ACM, 2019, doi:10.1145/3339471. short: C. Lenzen, J. Rybicki, Journal of the ACM 66 (2019). corr_author: '1' date_created: 2019-10-24T17:12:48Z date_published: 2019-09-01T00:00:00Z date_updated: 2025-04-14T07:44:06Z day: '01' ddc: - '000' department: - _id: DaAl doi: 10.1145/3339471 ec_funded: 1 external_id: arxiv: - '1705.06173' isi: - '000496514100001' file: - access_level: open_access checksum: 7e5d95c478e0e393f4927fcf7e48194e content_type: application/pdf creator: dernst date_created: 2019-10-25T12:58:38Z date_updated: 2020-07-14T12:47:46Z file_id: '6975' file_name: 2019_JACM_Lenzen.pdf file_size: 2183085 relation: main_file file_date_updated: 2020-07-14T12:47:46Z has_accepted_license: '1' intvolume: ' 66' isi: 1 issue: '5' language: - iso: eng month: '09' oa: 1 oa_version: Published Version project: - _id: 260C2330-B435-11E9-9278-68D0E5697425 call_identifier: H2020 grant_number: '754411' name: ISTplus - Postdoctoral Fellowships publication: Journal of the ACM publication_identifier: issn: - 0004-5411 publication_status: published publisher: ACM quality_controlled: '1' scopus_import: '1' status: public title: Self-stabilising Byzantine clock synchronisation is almost as easy as consensus tmp: image: /images/cc_by.png legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0) short: CC BY (4.0) type: journal_article user_id: 4359f0d1-fa6c-11eb-b949-802e58b17ae8 volume: 66 year: '2019' ... --- _id: '7122' abstract: - lang: eng text: Data-rich applications in machine-learning and control have motivated an intense research on large-scale optimization. Novel algorithms have been proposed and shown to have optimal convergence rates in terms of iteration counts. However, their practical performance is severely degraded by the cost of exchanging high-dimensional gradient vectors between computing nodes. Several gradient compression heuristics have recently been proposed to reduce communications, but few theoretical results exist that quantify how they impact algorithm convergence. This paper establishes and strengthens the convergence guarantees for gradient descent under a family of gradient compression techniques. For convex optimization problems, we derive admissible step sizes and quantify both the number of iterations and the number of bits that need to be exchanged to reach a target accuracy. Finally, we validate the performance of different gradient compression techniques in simulations. The numerical results highlight the properties of different gradient compression algorithms and confirm that fast convergence with limited information exchange is possible. article_number: '8619625' article_processing_charge: No author: - first_name: Sarit full_name: Khirirat, Sarit last_name: Khirirat - first_name: Mikael full_name: Johansson, Mikael last_name: Johansson - first_name: Dan-Adrian full_name: Alistarh, Dan-Adrian id: 4A899BFC-F248-11E8-B48F-1D18A9856A87 last_name: Alistarh orcid: 0000-0003-3650-940X citation: ama: 'Khirirat S, Johansson M, Alistarh D-A. Gradient compression for communication-limited convex optimization. In: 2018 IEEE Conference on Decision and Control. IEEE; 2019. doi:10.1109/cdc.2018.8619625' apa: 'Khirirat, S., Johansson, M., & Alistarh, D.-A. (2019). Gradient compression for communication-limited convex optimization. In 2018 IEEE Conference on Decision and Control. Miami Beach, FL, United States: IEEE. https://doi.org/10.1109/cdc.2018.8619625' chicago: Khirirat, Sarit, Mikael Johansson, and Dan-Adrian Alistarh. “Gradient Compression for Communication-Limited Convex Optimization.” In 2018 IEEE Conference on Decision and Control. IEEE, 2019. https://doi.org/10.1109/cdc.2018.8619625. ieee: S. Khirirat, M. Johansson, and D.-A. Alistarh, “Gradient compression for communication-limited convex optimization,” in 2018 IEEE Conference on Decision and Control, Miami Beach, FL, United States, 2019. ista: 'Khirirat S, Johansson M, Alistarh D-A. 2019. Gradient compression for communication-limited convex optimization. 2018 IEEE Conference on Decision and Control. CDC: Conference on Decision and Control, 8619625.' mla: Khirirat, Sarit, et al. “Gradient Compression for Communication-Limited Convex Optimization.” 2018 IEEE Conference on Decision and Control, 8619625, IEEE, 2019, doi:10.1109/cdc.2018.8619625. short: S. Khirirat, M. Johansson, D.-A. Alistarh, in:, 2018 IEEE Conference on Decision and Control, IEEE, 2019. conference: end_date: 2018-12-19 location: Miami Beach, FL, United States name: 'CDC: Conference on Decision and Control' start_date: 2018-12-17 date_created: 2019-11-26T15:07:49Z date_published: 2019-01-21T00:00:00Z date_updated: 2023-09-06T11:14:55Z day: '21' department: - _id: DaAl doi: 10.1109/cdc.2018.8619625 external_id: isi: - '000458114800023' isi: 1 language: - iso: eng month: '01' oa_version: None publication: 2018 IEEE Conference on Decision and Control publication_identifier: isbn: - '9781538613955' issn: - 0743-1546 publication_status: published publisher: IEEE quality_controlled: '1' scopus_import: '1' status: public title: Gradient compression for communication-limited convex optimization type: conference user_id: c635000d-4b10-11ee-a964-aac5a93f6ac1 year: '2019' ... --- _id: '7201' abstract: - lang: eng text: Applying machine learning techniques to the quickly growing data in science and industry requires highly-scalable algorithms. Large datasets are most commonly processed "data parallel" distributed across many nodes. Each node's contribution to the overall gradient is summed using a global allreduce. This allreduce is the single communication and thus scalability bottleneck for most machine learning workloads. We observe that frequently, many gradient values are (close to) zero, leading to sparse of sparsifyable communications. To exploit this insight, we analyze, design, and implement a set of communication-efficient protocols for sparse input data, in conjunction with efficient machine learning algorithms which can leverage these primitives. Our communication protocols generalize standard collective operations, by allowing processes to contribute arbitrary sparse input data vectors. Our generic communication library, SparCML1, extends MPI to support additional features, such as non-blocking (asynchronous) operations and low-precision data representations. As such, SparCML and its techniques will form the basis of future highly-scalable machine learning frameworks. article_number: a11 article_processing_charge: No arxiv: 1 author: - first_name: Cedric full_name: Renggli, Cedric last_name: Renggli - first_name: Saleh full_name: Ashkboos, Saleh id: 0D0A9058-257B-11EA-A937-9341C3D8BC8A last_name: Ashkboos - first_name: Mehdi full_name: Aghagolzadeh, Mehdi last_name: Aghagolzadeh - first_name: Dan-Adrian full_name: Alistarh, Dan-Adrian id: 4A899BFC-F248-11E8-B48F-1D18A9856A87 last_name: Alistarh orcid: 0000-0003-3650-940X - first_name: Torsten full_name: Hoefler, Torsten last_name: Hoefler citation: ama: 'Renggli C, Ashkboos S, Aghagolzadeh M, Alistarh D-A, Hoefler T. SparCML: High-performance sparse communication for machine learning. In: International Conference for High Performance Computing, Networking, Storage and Analysis, SC. ACM; 2019. doi:10.1145/3295500.3356222' apa: 'Renggli, C., Ashkboos, S., Aghagolzadeh, M., Alistarh, D.-A., & Hoefler, T. (2019). SparCML: High-performance sparse communication for machine learning. In International Conference for High Performance Computing, Networking, Storage and Analysis, SC. Denver, CO, Unites States: ACM. https://doi.org/10.1145/3295500.3356222' chicago: 'Renggli, Cedric, Saleh Ashkboos, Mehdi Aghagolzadeh, Dan-Adrian Alistarh, and Torsten Hoefler. “SparCML: High-Performance Sparse Communication for Machine Learning.” In International Conference for High Performance Computing, Networking, Storage and Analysis, SC. ACM, 2019. https://doi.org/10.1145/3295500.3356222.' ieee: 'C. Renggli, S. Ashkboos, M. Aghagolzadeh, D.-A. Alistarh, and T. Hoefler, “SparCML: High-performance sparse communication for machine learning,” in International Conference for High Performance Computing, Networking, Storage and Analysis, SC, Denver, CO, Unites States, 2019.' ista: 'Renggli C, Ashkboos S, Aghagolzadeh M, Alistarh D-A, Hoefler T. 2019. SparCML: High-performance sparse communication for machine learning. International Conference for High Performance Computing, Networking, Storage and Analysis, SC. SC: Conference for High Performance Computing, Networking, Storage and Analysis, a11.' mla: 'Renggli, Cedric, et al. “SparCML: High-Performance Sparse Communication for Machine Learning.” International Conference for High Performance Computing, Networking, Storage and Analysis, SC, a11, ACM, 2019, doi:10.1145/3295500.3356222.' short: C. Renggli, S. Ashkboos, M. Aghagolzadeh, D.-A. Alistarh, T. Hoefler, in:, International Conference for High Performance Computing, Networking, Storage and Analysis, SC, ACM, 2019. conference: end_date: 2019-11-19 location: Denver, CO, Unites States name: 'SC: Conference for High Performance Computing, Networking, Storage and Analysis' start_date: 2019-11-17 date_created: 2019-12-22T23:00:42Z date_published: 2019-11-17T00:00:00Z date_updated: 2025-07-10T11:54:21Z day: '17' department: - _id: DaAl doi: 10.1145/3295500.3356222 ec_funded: 1 external_id: arxiv: - '1802.08021' isi: - '000545976800011' isi: 1 language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/1802.08021 month: '11' oa: 1 oa_version: Preprint project: - _id: 268A44D6-B435-11E9-9278-68D0E5697425 call_identifier: H2020 grant_number: '805223' name: Elastic Coordination for Scalable Machine Learning publication: International Conference for High Performance Computing, Networking, Storage and Analysis, SC publication_identifier: eissn: - 2167-4337 isbn: - '9781450362290' issn: - 2167-4329 publication_status: published publisher: ACM quality_controlled: '1' scopus_import: '1' status: public title: 'SparCML: High-performance sparse communication for machine learning' type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 year: '2019' ... --- _id: '7214' abstract: - lang: eng text: "Background: Many cancer genomes are extensively rearranged with highly aberrant chromosomal karyotypes. Structural and copy number variations in cancer genomes can be determined via abnormal mapping of sequenced reads to the reference genome. Recently it became possible to reconcile both of these types of large-scale variations into a karyotype graph representation of the rearranged cancer genomes. Such a representation, however, does not directly describe the linear and/or circular structure of the underlying rearranged cancer chromosomes, thus limiting possible analysis of cancer genomes somatic evolutionary process as well as functional genomic changes brought by the large-scale genome rearrangements.\r\n\r\nResults: Here we address the aforementioned limitation by introducing a novel methodological framework for recovering rearranged cancer chromosomes from karyotype graphs. For a cancer karyotype graph we formulate an Eulerian Decomposition Problem (EDP) of finding a collection of linear and/or circular rearranged cancer chromosomes that are determined by the graph. We derive and prove computational complexities for several variations of the EDP. We then demonstrate that Eulerian decomposition of the cancer karyotype graphs is not always unique and present the Consistent Contig Covering Problem (CCCP) of recovering unambiguous cancer contigs from the cancer karyotype graph, and describe a novel algorithm CCR capable of solving CCCP in polynomial time. We apply CCR on a prostate cancer dataset and demonstrate that it is capable of consistently recovering large cancer contigs even when underlying cancer genomes are highly rearranged.\r\n\r\nConclusions: CCR can recover rearranged cancer contigs from karyotype graphs thereby addressing existing limitation in inferring chromosomal structures of rearranged cancer genomes and advancing our understanding of both patient/cancer-specific as well as the overall genetic instability in cancer." article_number: '641' article_processing_charge: No article_type: original author: - first_name: Sergey full_name: Aganezov, Sergey last_name: Aganezov - first_name: Ilya full_name: Zban, Ilya last_name: Zban - first_name: Vitalii full_name: Aksenov, Vitalii id: 2980135A-F248-11E8-B48F-1D18A9856A87 last_name: Aksenov - first_name: Nikita full_name: Alexeev, Nikita last_name: Alexeev - first_name: Michael C. full_name: Schatz, Michael C. last_name: Schatz citation: ama: Aganezov S, Zban I, Aksenov V, Alexeev N, Schatz MC. Recovering rearranged cancer chromosomes from karyotype graphs. BMC Bioinformatics. 2019;20. doi:10.1186/s12859-019-3208-4 apa: Aganezov, S., Zban, I., Aksenov, V., Alexeev, N., & Schatz, M. C. (2019). Recovering rearranged cancer chromosomes from karyotype graphs. BMC Bioinformatics. BMC. https://doi.org/10.1186/s12859-019-3208-4 chicago: Aganezov, Sergey, Ilya Zban, Vitalii Aksenov, Nikita Alexeev, and Michael C. Schatz. “Recovering Rearranged Cancer Chromosomes from Karyotype Graphs.” BMC Bioinformatics. BMC, 2019. https://doi.org/10.1186/s12859-019-3208-4. ieee: S. Aganezov, I. Zban, V. Aksenov, N. Alexeev, and M. C. Schatz, “Recovering rearranged cancer chromosomes from karyotype graphs,” BMC Bioinformatics, vol. 20. BMC, 2019. ista: Aganezov S, Zban I, Aksenov V, Alexeev N, Schatz MC. 2019. Recovering rearranged cancer chromosomes from karyotype graphs. BMC Bioinformatics. 20, 641. mla: Aganezov, Sergey, et al. “Recovering Rearranged Cancer Chromosomes from Karyotype Graphs.” BMC Bioinformatics, vol. 20, 641, BMC, 2019, doi:10.1186/s12859-019-3208-4. short: S. Aganezov, I. Zban, V. Aksenov, N. Alexeev, M.C. Schatz, BMC Bioinformatics 20 (2019). date_created: 2019-12-29T23:00:46Z date_published: 2019-12-17T00:00:00Z date_updated: 2026-04-16T08:35:00Z day: '17' ddc: - '570' department: - _id: DaAl doi: 10.1186/s12859-019-3208-4 external_id: isi: - '000511618800007' file: - access_level: open_access checksum: 7a30357efdcf8f66587ed495c0927724 content_type: application/pdf creator: dernst date_created: 2020-01-02T16:10:58Z date_updated: 2020-07-14T12:47:54Z file_id: '7221' file_name: 2019_BMCBioinfo_Aganezov.pdf file_size: 1917374 relation: main_file file_date_updated: 2020-07-14T12:47:54Z has_accepted_license: '1' intvolume: ' 20' isi: 1 language: - iso: eng month: '12' oa: 1 oa_version: Published Version publication: BMC Bioinformatics publication_identifier: eissn: - 1471-2105 publication_status: published publisher: BMC quality_controlled: '1' scopus_import: '1' status: public title: Recovering rearranged cancer chromosomes from karyotype graphs tmp: image: /images/cc_by.png legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0) short: CC BY (4.0) type: journal_article user_id: ba8df636-2132-11f1-aed0-ed93e2281fdd volume: 20 year: '2019' ... --- _id: '7228' abstract: - lang: eng text: "Traditional concurrent programming involves manipulating shared mutable state. Alternatives to this programming style are communicating sequential processes (CSP) and actor models, which share data via explicit communication. These models have been known for almost half a century, and have recently had started to gain significant traction among modern programming languages. The common abstraction for communication between several processes is the channel. Although channels are similar to producer-consumer data structures, they have different semantics and support additional operations, such as the select expression. Despite their growing popularity, most known implementations of channels use lock-based data structures and can be rather inefficient.\r\n\r\nIn this paper, we present the first efficient lock-free algorithm for implementing a communication channel for CSP programming. We provide implementations and experimental results in the Kotlin and Go programming languages. Our new algorithm outperforms existing implementations on many workloads, while providing non-blocking progress guarantee. Our design can serve as an example of how to construct general communication data structures for CSP and actor models. " alternative_title: - LNCS article_processing_charge: No author: - first_name: Nikita full_name: Koval, Nikita id: 2F4DB10C-F248-11E8-B48F-1D18A9856A87 last_name: Koval - first_name: Dan-Adrian full_name: Alistarh, Dan-Adrian id: 4A899BFC-F248-11E8-B48F-1D18A9856A87 last_name: Alistarh orcid: 0000-0003-3650-940X - first_name: Roman full_name: Elizarov, Roman last_name: Elizarov citation: ama: 'Koval N, Alistarh D-A, Elizarov R. Scalable FIFO channels for programming via communicating sequential processes. In: 25th Anniversary of Euro-Par. Vol 11725. Springer Nature; 2019:317-333. doi:10.1007/978-3-030-29400-7_23' apa: 'Koval, N., Alistarh, D.-A., & Elizarov, R. (2019). Scalable FIFO channels for programming via communicating sequential processes. In 25th Anniversary of Euro-Par (Vol. 11725, pp. 317–333). Göttingen, Germany: Springer Nature. https://doi.org/10.1007/978-3-030-29400-7_23' chicago: Koval, Nikita, Dan-Adrian Alistarh, and Roman Elizarov. “Scalable FIFO Channels for Programming via Communicating Sequential Processes.” In 25th Anniversary of Euro-Par, 11725:317–33. Springer Nature, 2019. https://doi.org/10.1007/978-3-030-29400-7_23. ieee: N. Koval, D.-A. Alistarh, and R. Elizarov, “Scalable FIFO channels for programming via communicating sequential processes,” in 25th Anniversary of Euro-Par, Göttingen, Germany, 2019, vol. 11725, pp. 317–333. ista: 'Koval N, Alistarh D-A, Elizarov R. 2019. Scalable FIFO channels for programming via communicating sequential processes. 25th Anniversary of Euro-Par. Euro-Par: European Conference on Parallel Processing, LNCS, vol. 11725, 317–333.' mla: Koval, Nikita, et al. “Scalable FIFO Channels for Programming via Communicating Sequential Processes.” 25th Anniversary of Euro-Par, vol. 11725, Springer Nature, 2019, pp. 317–33, doi:10.1007/978-3-030-29400-7_23. short: N. Koval, D.-A. Alistarh, R. Elizarov, in:, 25th Anniversary of Euro-Par, Springer Nature, 2019, pp. 317–333. conference: end_date: 2019-08-30 location: Göttingen, Germany name: 'Euro-Par: European Conference on Parallel Processing' start_date: 2019-08-26 date_created: 2020-01-05T23:00:46Z date_published: 2019-08-13T00:00:00Z date_updated: 2023-09-06T14:53:59Z day: '13' department: - _id: DaAl doi: 10.1007/978-3-030-29400-7_23 external_id: isi: - '000851061400023' intvolume: ' 11725' isi: 1 language: - iso: eng month: '08' oa_version: None page: 317-333 publication: 25th Anniversary of Euro-Par publication_identifier: eissn: - 1611-3349 isbn: - 978-3-0302-9399-4 issn: - 0302-9743 publication_status: published publisher: Springer Nature quality_controlled: '1' scopus_import: '1' status: public title: Scalable FIFO channels for programming via communicating sequential processes type: conference user_id: c635000d-4b10-11ee-a964-aac5a93f6ac1 volume: 11725 year: '2019' ... --- _id: '7542' abstract: - lang: eng text: We present a novel class of convolutional neural networks (CNNs) for set functions,i.e., data indexed with the powerset of a finite set. The convolutions are derivedas linear, shift-equivariant functions for various notions of shifts on set functions.The framework is fundamentally different from graph convolutions based on theLaplacian, as it provides not one but several basic shifts, one for each element inthe ground set. Prototypical experiments with several set function classificationtasks on synthetic datasets and on datasets derived from real-world hypergraphsdemonstrate the potential of our new powerset CNNs. article_processing_charge: No arxiv: 1 author: - first_name: Chris full_name: Wendler, Chris last_name: Wendler - first_name: Dan-Adrian full_name: Alistarh, Dan-Adrian id: 4A899BFC-F248-11E8-B48F-1D18A9856A87 last_name: Alistarh orcid: 0000-0003-3650-940X - first_name: Markus full_name: Püschel, Markus last_name: Püschel citation: ama: 'Wendler C, Alistarh D-A, Püschel M. Powerset convolutional neural networks. In: Vol 32. Neural Information Processing Systems Foundation; 2019:927-938.' apa: 'Wendler, C., Alistarh, D.-A., & Püschel, M. (2019). Powerset convolutional neural networks (Vol. 32, pp. 927–938). Presented at the NIPS: Conference on Neural Information Processing Systems, Vancouver, Canada: Neural Information Processing Systems Foundation.' chicago: Wendler, Chris, Dan-Adrian Alistarh, and Markus Püschel. “Powerset Convolutional Neural Networks,” 32:927–38. Neural Information Processing Systems Foundation, 2019. ieee: 'C. Wendler, D.-A. Alistarh, and M. Püschel, “Powerset convolutional neural networks,” presented at the NIPS: Conference on Neural Information Processing Systems, Vancouver, Canada, 2019, vol. 32, pp. 927–938.' ista: 'Wendler C, Alistarh D-A, Püschel M. 2019. Powerset convolutional neural networks. NIPS: Conference on Neural Information Processing Systems vol. 32, 927–938.' mla: Wendler, Chris, et al. Powerset Convolutional Neural Networks. Vol. 32, Neural Information Processing Systems Foundation, 2019, pp. 927–38. short: C. Wendler, D.-A. Alistarh, M. Püschel, in:, Neural Information Processing Systems Foundation, 2019, pp. 927–938. conference: end_date: 2019-12-14 location: Vancouver, Canada name: 'NIPS: Conference on Neural Information Processing Systems' start_date: 2019-12-08 date_created: 2020-02-28T10:03:24Z date_published: 2019-12-01T00:00:00Z date_updated: 2026-06-18T19:23:08Z day: '01' ddc: - '000' department: - _id: DaAl ec_funded: 1 external_id: arxiv: - '1909.02253' isi: - '000534424300084' intvolume: ' 32' isi: 1 language: - iso: eng main_file_link: - open_access: '1' url: http://papers.nips.cc/paper/8379-powerset-convolutional-neural-networks month: '12' oa: 1 oa_version: Published Version page: 927-938 project: - _id: 268A44D6-B435-11E9-9278-68D0E5697425 call_identifier: H2020 grant_number: '805223' name: Elastic Coordination for Scalable Machine Learning publication_identifier: issn: - 1049-5258 publication_status: published publisher: Neural Information Processing Systems Foundation quality_controlled: '1' status: public title: Powerset convolutional neural networks type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 32 year: '2019' ... --- _id: '7437' abstract: - lang: eng text: 'Most of today''s distributed machine learning systems assume reliable networks: whenever two machines exchange information (e.g., gradients or models), the network should guarantee the delivery of the message. At the same time, recent work exhibits the impressive tolerance of machine learning algorithms to errors or noise arising from relaxed communication or synchronization. In this paper, we connect these two trends, and consider the following question: Can we design machine learning systems that are tolerant to network unreliability during training? With this motivation, we focus on a theoretical problem of independent interest-given a standard distributed parameter server architecture, if every communication between the worker and the server has a non-zero probability p of being dropped, does there exist an algorithm that still converges, and at what speed? The technical contribution of this paper is a novel theoretical analysis proving that distributed learning over unreliable network can achieve comparable convergence rate to centralized or distributed learning over reliable networks. Further, we prove that the influence of the packet drop rate diminishes with the growth of the number of parameter servers. We map this theoretical result onto a real-world scenario, training deep neural networks over an unreliable network layer, and conduct network simulation to validate the system improvement by allowing the networks to be unreliable.' article_processing_charge: No arxiv: 1 author: - first_name: Chen full_name: Yu, Chen last_name: Yu - first_name: Hanlin full_name: Tang, Hanlin last_name: Tang - first_name: Cedric full_name: Renggli, Cedric last_name: Renggli - first_name: Simon full_name: Kassing, Simon last_name: Kassing - first_name: Ankit full_name: Singla, Ankit last_name: Singla - first_name: Dan-Adrian full_name: Alistarh, Dan-Adrian id: 4A899BFC-F248-11E8-B48F-1D18A9856A87 last_name: Alistarh orcid: 0000-0003-3650-940X - first_name: Ce full_name: Zhang, Ce last_name: Zhang - first_name: Ji full_name: Liu, Ji last_name: Liu citation: ama: 'Yu C, Tang H, Renggli C, et al. Distributed learning over unreliable networks. In: 36th International Conference on Machine Learning. Vol 2019-June. IMLS; 2019:12481-12512.' apa: 'Yu, C., Tang, H., Renggli, C., Kassing, S., Singla, A., Alistarh, D.-A., … Liu, J. (2019). Distributed learning over unreliable networks. In 36th International Conference on Machine Learning (Vol. 2019–June, pp. 12481–12512). Long Beach, CA, United States: IMLS.' chicago: Yu, Chen, Hanlin Tang, Cedric Renggli, Simon Kassing, Ankit Singla, Dan-Adrian Alistarh, Ce Zhang, and Ji Liu. “Distributed Learning over Unreliable Networks.” In 36th International Conference on Machine Learning, 2019–June:12481–512. IMLS, 2019. ieee: C. Yu et al., “Distributed learning over unreliable networks,” in 36th International Conference on Machine Learning, Long Beach, CA, United States, 2019, vol. 2019–June, pp. 12481–12512. ista: 'Yu C, Tang H, Renggli C, Kassing S, Singla A, Alistarh D-A, Zhang C, Liu J. 2019. Distributed learning over unreliable networks. 36th International Conference on Machine Learning. ICML: International Conference on Machine Learning vol. 2019–June, 12481–12512.' mla: Yu, Chen, et al. “Distributed Learning over Unreliable Networks.” 36th International Conference on Machine Learning, vol. 2019–June, IMLS, 2019, pp. 12481–512. short: C. Yu, H. Tang, C. Renggli, S. Kassing, A. Singla, D.-A. Alistarh, C. Zhang, J. Liu, in:, 36th International Conference on Machine Learning, IMLS, 2019, pp. 12481–12512. conference: end_date: 2019-06-15 location: Long Beach, CA, United States name: 'ICML: International Conference on Machine Learning' start_date: 2019-06-10 das_tickbox: '1' date_created: 2020-02-02T23:01:06Z date_published: 2019-06-01T00:00:00Z date_updated: 2026-07-07T13:41:57Z day: '01' department: - _id: DaAl external_id: arxiv: - '1810.07766' isi: - '000684034307036' isi: 1 language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/1810.07766 month: '06' oa: 1 oa_version: Preprint page: 12481-12512 publication: 36th International Conference on Machine Learning publication_identifier: isbn: - '9781510886988' publication_status: published publisher: IMLS quality_controlled: '1' scopus_import: '1' status: public title: Distributed learning over unreliable networks type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 2019-June year: '2019' ... --- OA_place: publisher OA_type: free access _id: '5947' abstract: - lang: eng text: Graph algorithms applied in many applications, including social networks, communication networks, VLSI design, graphics, and several others, require dynamic modifications - addition and removal of vertices and/or edges - in the graph. This paper presents a novel concurrent non-blocking algorithm to implement a dynamic unbounded directed graph in a shared-memory machine. The addition and removal operations of vertices and edges are lock-free. For a finite sized graph, the lookup operations are wait-free. Most significant component of the presented algorithm is the reachability query in a concurrent graph. The reachability queries in our algorithm are obstruction-free and thus impose minimal additional synchronization cost over other operations. We prove that each of the data structure operations are linearizable. We extensively evaluate a sample C/C++ implementation of the algorithm through a number of micro-benchmarks. The experimental results show that the proposed algorithm scales well with the number of threads and on an average provides 5 to 7x performance improvement over a concurrent graph implementation using coarse-grained locking. article_processing_charge: No arxiv: 1 author: - first_name: Bapi full_name: Chatterjee, Bapi id: 3C41A08A-F248-11E8-B48F-1D18A9856A87 last_name: Chatterjee orcid: 0000-0002-2742-4028 - first_name: Sathya full_name: Peri, Sathya last_name: Peri - first_name: Muktikanta full_name: Sa, Muktikanta last_name: Sa - first_name: Nandini full_name: Singhal, Nandini last_name: Singhal citation: ama: 'Chatterjee B, Peri S, Sa M, Singhal N. A simple and practical concurrent non-blocking unbounded graph with linearizable reachability queries. In: ACM International Conference Proceeding Series. ACM; 2019:168-177. doi:10.1145/3288599.3288617' apa: 'Chatterjee, B., Peri, S., Sa, M., & Singhal, N. (2019). A simple and practical concurrent non-blocking unbounded graph with linearizable reachability queries. In ACM International Conference Proceeding Series (pp. 168–177). Bangalore, India: ACM. https://doi.org/10.1145/3288599.3288617' chicago: Chatterjee, Bapi, Sathya Peri, Muktikanta Sa, and Nandini Singhal. “A Simple and Practical Concurrent Non-Blocking Unbounded Graph with Linearizable Reachability Queries.” In ACM International Conference Proceeding Series, 168–77. ACM, 2019. https://doi.org/10.1145/3288599.3288617. ieee: B. Chatterjee, S. Peri, M. Sa, and N. Singhal, “A simple and practical concurrent non-blocking unbounded graph with linearizable reachability queries,” in ACM International Conference Proceeding Series, Bangalore, India, 2019, pp. 168–177. ista: 'Chatterjee B, Peri S, Sa M, Singhal N. 2019. A simple and practical concurrent non-blocking unbounded graph with linearizable reachability queries. ACM International Conference Proceeding Series. ICDCN: Conference on Distributed Computing and Networking, 168–177.' mla: Chatterjee, Bapi, et al. “A Simple and Practical Concurrent Non-Blocking Unbounded Graph with Linearizable Reachability Queries.” ACM International Conference Proceeding Series, ACM, 2019, pp. 168–77, doi:10.1145/3288599.3288617. short: B. Chatterjee, S. Peri, M. Sa, N. Singhal, in:, ACM International Conference Proceeding Series, ACM, 2019, pp. 168–177. conference: end_date: 2019-01-07 location: Bangalore, India name: 'ICDCN: Conference on Distributed Computing and Networking' start_date: 2019-01-04 corr_author: '1' date_created: 2019-02-10T22:59:17Z date_published: 2019-01-04T00:00:00Z date_updated: 2026-07-28T13:34:36Z day: '04' department: - _id: DaAl doi: 10.1145/3288599.3288617 external_id: arxiv: - '1809.00896' isi: - '000484491600019' isi: 1 language: - iso: eng main_file_link: - open_access: '1' url: https://doi.org/10.1145/3288599.3288617 month: '01' oa: 1 oa_version: Published Version page: 168-177 publication: ACM International Conference Proceeding Series publication_identifier: isbn: - '978-1-4503-6094-4 ' publication_status: published publisher: ACM quality_controlled: '1' scopus_import: '1' status: public title: A simple and practical concurrent non-blocking unbounded graph with linearizable reachability queries type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 year: '2019' ... --- _id: '536' abstract: - lang: eng text: 'We consider the problem of consensus in the challenging classic model. In this model, the adversary is adaptive; it can choose which processors crash at any point during the course of the algorithm. Further, communication is via asynchronous message passing: there is no known upper bound on the time to send a message from one processor to another, and all messages and coin flips are seen by the adversary. We describe a new randomized consensus protocol with expected message complexity O(n2log2n) when fewer than n / 2 processes may fail by crashing. This is an almost-linear improvement over the best previously known protocol, and within logarithmic factors of a known Ω(n2) message lower bound. The protocol further ensures that no process sends more than O(nlog3n) messages in expectation, which is again within logarithmic factors of optimal. We also present a generalization of the algorithm to an arbitrary number of failures t, which uses expected O(nt+t2log2t) total messages. Our approach is to build a message-efficient, resilient mechanism for aggregating individual processor votes, implementing the message-passing equivalent of a weak shared coin. Roughly, in our protocol, a processor first announces its votes to small groups, then propagates them to increasingly larger groups as it generates more and more votes. To bound the number of messages that an individual process might have to send or receive, the protocol progressively increases the weight of generated votes. The main technical challenge is bounding the impact of votes that are still “in flight” (generated, but not fully propagated) on the final outcome of the shared coin, especially since such votes might have different weights. We achieve this by leveraging the structure of the algorithm, and a technical argument based on martingale concentration bounds. Overall, we show that it is possible to build an efficient message-passing implementation of a shared coin, and in the process (almost-optimally) solve the classic consensus problem in the asynchronous message-passing model.' article_processing_charge: Yes (via OA deal) author: - first_name: Dan-Adrian full_name: Alistarh, Dan-Adrian id: 4A899BFC-F248-11E8-B48F-1D18A9856A87 last_name: Alistarh orcid: 0000-0003-3650-940X - first_name: James full_name: Aspnes, James last_name: Aspnes - first_name: Valerie full_name: King, Valerie last_name: King - first_name: Jared full_name: Saia, Jared last_name: Saia citation: ama: Alistarh D-A, Aspnes J, King V, Saia J. Communication-efficient randomized consensus. Distributed Computing. 2018;31(6):489-501. doi:10.1007/s00446-017-0315-1 apa: Alistarh, D.-A., Aspnes, J., King, V., & Saia, J. (2018). Communication-efficient randomized consensus. Distributed Computing. Springer. https://doi.org/10.1007/s00446-017-0315-1 chicago: Alistarh, Dan-Adrian, James Aspnes, Valerie King, and Jared Saia. “Communication-Efficient Randomized Consensus.” Distributed Computing. Springer, 2018. https://doi.org/10.1007/s00446-017-0315-1. ieee: D.-A. Alistarh, J. Aspnes, V. King, and J. Saia, “Communication-efficient randomized consensus,” Distributed Computing, vol. 31, no. 6. Springer, pp. 489–501, 2018. ista: Alistarh D-A, Aspnes J, King V, Saia J. 2018. Communication-efficient randomized consensus. Distributed Computing. 31(6), 489–501. mla: Alistarh, Dan-Adrian, et al. “Communication-Efficient Randomized Consensus.” Distributed Computing, vol. 31, no. 6, Springer, 2018, pp. 489–501, doi:10.1007/s00446-017-0315-1. short: D.-A. Alistarh, J. Aspnes, V. King, J. Saia, Distributed Computing 31 (2018) 489–501. corr_author: '1' date_created: 2018-12-11T11:47:01Z date_published: 2018-11-01T00:00:00Z date_updated: 2026-04-16T09:53:54Z day: '01' ddc: - '000' department: - _id: DaAl doi: 10.1007/s00446-017-0315-1 external_id: isi: - '000443832300005' file: - access_level: open_access checksum: 69b46e537acdcac745237ddb853fcbb5 content_type: application/pdf creator: dernst date_created: 2019-01-22T07:25:51Z date_updated: 2020-07-14T12:46:38Z file_id: '5867' file_name: 2017_DistribComp_Alistarh.pdf file_size: 595707 relation: main_file file_date_updated: 2020-07-14T12:46:38Z has_accepted_license: '1' intvolume: ' 31' isi: 1 issue: '6' language: - iso: eng month: '11' oa: 1 oa_version: Published Version page: 489-501 project: - _id: B67AFEDC-15C9-11EA-A837-991A96BB2854 name: IST Austria Open Access Fund publication: Distributed Computing publication_identifier: issn: - 0178-2770 publication_status: published publisher: Springer publist_id: '7281' quality_controlled: '1' scopus_import: '1' status: public title: Communication-efficient randomized consensus tmp: image: /images/cc_by.png legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0) short: CC BY (4.0) type: journal_article user_id: ba8df636-2132-11f1-aed0-ed93e2281fdd volume: 31 year: '2018' ... --- _id: '5961' abstract: - lang: eng text: "The area of machine learning has made considerable progress over the past decade, enabled by the widespread availability of large datasets, as well as by improved algorithms and models. Given the large computational demands of machine learning workloads, parallelism, implemented either through single-node concurrency or through multi-node distribution, has been a third key ingredient to advances in machine learning.\r\nThe goal of this tutorial is to provide the audience with an overview of standard distribution techniques in machine learning, with an eye towards the intriguing trade-offs between synchronization and communication costs of distributed machine learning algorithms, on the one hand, and their convergence, on the other.The tutorial will focus on parallelization strategies for the fundamental stochastic gradient descent (SGD) algorithm, which is a key tool when training machine learning models, from classical instances such as linear regression, to state-of-the-art neural network architectures.\r\nThe tutorial will describe the guarantees provided by this algorithm in the sequential case, and then move on to cover both shared-memory and message-passing parallelization strategies, together with the guarantees they provide, and corresponding trade-offs. The presentation will conclude with a broad overview of ongoing research in distributed and concurrent machine learning. The tutorial will assume no prior knowledge beyond familiarity with basic concepts in algebra and analysis.\r\n" article_processing_charge: No author: - first_name: Dan-Adrian full_name: Alistarh, Dan-Adrian id: 4A899BFC-F248-11E8-B48F-1D18A9856A87 last_name: Alistarh orcid: 0000-0003-3650-940X citation: ama: 'Alistarh D-A. A brief tutorial on distributed and concurrent machine learning. In: Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18. ACM; 2018:487-488. doi:10.1145/3212734.3212798' apa: 'Alistarh, D.-A. (2018). A brief tutorial on distributed and concurrent machine learning. In Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18 (pp. 487–488). Egham, United Kingdom: ACM. https://doi.org/10.1145/3212734.3212798' chicago: Alistarh, Dan-Adrian. “A Brief Tutorial on Distributed and Concurrent Machine Learning.” In Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18, 487–88. ACM, 2018. https://doi.org/10.1145/3212734.3212798. ieee: D.-A. Alistarh, “A brief tutorial on distributed and concurrent machine learning,” in Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18, Egham, United Kingdom, 2018, pp. 487–488. ista: 'Alistarh D-A. 2018. A brief tutorial on distributed and concurrent machine learning. Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18. PODC: Principles of Distributed Computing, 487–488.' mla: Alistarh, Dan-Adrian. “A Brief Tutorial on Distributed and Concurrent Machine Learning.” Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18, ACM, 2018, pp. 487–88, doi:10.1145/3212734.3212798. short: D.-A. Alistarh, in:, Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18, ACM, 2018, pp. 487–488. conference: end_date: 2018-07-27 location: Egham, United Kingdom name: 'PODC: Principles of Distributed Computing' start_date: 2018-07-23 date_created: 2019-02-13T09:48:55Z date_published: 2018-07-27T00:00:00Z date_updated: 2025-06-03T11:56:33Z day: '27' department: - _id: DaAl doi: 10.1145/3212734.3212798 external_id: isi: - '000458186900063' isi: 1 language: - iso: eng month: '07' oa_version: None page: 487-488 publication: Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing - PODC '18 publication_identifier: isbn: - '9781450357951' publication_status: published publisher: ACM quality_controlled: '1' scopus_import: '1' status: public title: A brief tutorial on distributed and concurrent machine learning type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 year: '2018' ... --- _id: '5962' abstract: - lang: eng text: Stochastic Gradient Descent (SGD) is a fundamental algorithm in machine learning, representing the optimization backbone for training several classic models, from regression to neural networks. Given the recent practical focus on distributed machine learning, significant work has been dedicated to the convergence properties of this algorithm under the inconsistent and noisy updates arising from execution in a distributed environment. However, surprisingly, the convergence properties of this classic algorithm in the standard shared-memory model are still not well-understood. In this work, we address this gap, and provide new convergence bounds for lock-free concurrent stochastic gradient descent, executing in the classic asynchronous shared memory model, against a strong adaptive adversary. Our results give improved upper and lower bounds on the "price of asynchrony'' when executing the fundamental SGD algorithm in a concurrent setting. They show that this classic optimization tool can converge faster and with a wider range of parameters than previously known under asynchronous iterations. At the same time, we exhibit a fundamental trade-off between the maximum delay in the system and the rate at which SGD can converge, which governs the set of parameters under which this algorithm can still work efficiently. article_processing_charge: No arxiv: 1 author: - first_name: Dan-Adrian full_name: Alistarh, Dan-Adrian id: 4A899BFC-F248-11E8-B48F-1D18A9856A87 last_name: Alistarh orcid: 0000-0003-3650-940X - first_name: Christopher full_name: De Sa, Christopher last_name: De Sa - first_name: Nikola H full_name: Konstantinov, Nikola H id: 4B9D76E4-F248-11E8-B48F-1D18A9856A87 last_name: Konstantinov citation: ama: 'Alistarh D-A, De Sa C, Konstantinov NH. The convergence of stochastic gradient descent in asynchronous shared memory. In: Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18. ACM; 2018:169-178. doi:10.1145/3212734.3212763' apa: 'Alistarh, D.-A., De Sa, C., & Konstantinov, N. H. (2018). The convergence of stochastic gradient descent in asynchronous shared memory. In Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18 (pp. 169–178). Egham, United Kingdom: ACM. https://doi.org/10.1145/3212734.3212763' chicago: Alistarh, Dan-Adrian, Christopher De Sa, and Nikola H Konstantinov. “The Convergence of Stochastic Gradient Descent in Asynchronous Shared Memory.” In Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18, 169–78. ACM, 2018. https://doi.org/10.1145/3212734.3212763. ieee: D.-A. Alistarh, C. De Sa, and N. H. Konstantinov, “The convergence of stochastic gradient descent in asynchronous shared memory,” in Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18, Egham, United Kingdom, 2018, pp. 169–178. ista: 'Alistarh D-A, De Sa C, Konstantinov NH. 2018. The convergence of stochastic gradient descent in asynchronous shared memory. Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18. PODC: Principles of Distributed Computing, 169–178.' mla: Alistarh, Dan-Adrian, et al. “The Convergence of Stochastic Gradient Descent in Asynchronous Shared Memory.” Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18, ACM, 2018, pp. 169–78, doi:10.1145/3212734.3212763. short: D.-A. Alistarh, C. De Sa, N.H. Konstantinov, in:, Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18, ACM, 2018, pp. 169–178. conference: end_date: 2018-07-27 location: Egham, United Kingdom name: 'PODC: Principles of Distributed Computing' start_date: 2018-07-23 date_created: 2019-02-13T09:58:58Z date_published: 2018-07-23T00:00:00Z date_updated: 2025-06-03T11:56:41Z day: '23' department: - _id: DaAl doi: 10.1145/3212734.3212763 external_id: arxiv: - '1803.08841' isi: - '000458186900022' isi: 1 language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/1803.08841 month: '07' oa: 1 oa_version: Preprint page: 169-178 publication: Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing - PODC '18 publication_identifier: isbn: - '9781450357951' publication_status: published publisher: ACM quality_controlled: '1' scopus_import: '1' status: public title: The convergence of stochastic gradient descent in asynchronous shared memory type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 year: '2018' ... --- _id: '5963' abstract: - lang: eng text: 'There has been significant progress in understanding the parallelism inherent to iterative sequential algorithms: for many classic algorithms, the depth of the dependence structure is now well understood, and scheduling techniques have been developed to exploit this shallow dependence structure for efficient parallel implementations. A related, applied research strand has studied methods by which certain iterative task-based algorithms can be efficiently parallelized via relaxed concurrent priority schedulers. These allow for high concurrency when inserting and removing tasks, at the cost of executing superfluous work due to the relaxed semantics of the scheduler. In this work, we take a step towards unifying these two research directions, by showing that there exists a family of relaxed priority schedulers that can efficiently and deterministically execute classic iterative algorithms such as greedy maximal independent set (MIS) and matching. Our primary result shows that, given a randomized scheduler with an expected relaxation factor of k in terms of the maximum allowed priority inversions on a task, and any graph on n vertices, the scheduler is able to execute greedy MIS with only an additive factor of \poly(k) expected additional iterations compared to an exact (but not scalable) scheduler. This counter-intuitive result demonstrates that the overhead of relaxation when computing MIS is not dependent on the input size or structure of the input graph. Experimental results show that this overhead can be clearly offset by the gain in performance due to the highly scalable scheduler. In sum, we present an efficient method to deterministically parallelize iterative sequential algorithms, with provable runtime guarantees in terms of the number of executed tasks to completion.' article_processing_charge: No arxiv: 1 author: - first_name: Dan-Adrian full_name: Alistarh, Dan-Adrian id: 4A899BFC-F248-11E8-B48F-1D18A9856A87 last_name: Alistarh orcid: 0000-0003-3650-940X - first_name: Trevor A full_name: Brown, Trevor A id: 3569F0A0-F248-11E8-B48F-1D18A9856A87 last_name: Brown - first_name: Justin full_name: Kopinsky, Justin last_name: Kopinsky - first_name: Giorgi full_name: Nadiradze, Giorgi last_name: Nadiradze citation: ama: 'Alistarh D-A, Brown TA, Kopinsky J, Nadiradze G. Relaxed schedulers can efficiently parallelize iterative algorithms. In: Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18. ACM; 2018:377-386. doi:10.1145/3212734.3212756' apa: 'Alistarh, D.-A., Brown, T. A., Kopinsky, J., & Nadiradze, G. (2018). Relaxed schedulers can efficiently parallelize iterative algorithms. In Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18 (pp. 377–386). Egham, United Kingdom: ACM. https://doi.org/10.1145/3212734.3212756' chicago: Alistarh, Dan-Adrian, Trevor A Brown, Justin Kopinsky, and Giorgi Nadiradze. “Relaxed Schedulers Can Efficiently Parallelize Iterative Algorithms.” In Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18, 377–86. ACM, 2018. https://doi.org/10.1145/3212734.3212756. ieee: D.-A. Alistarh, T. A. Brown, J. Kopinsky, and G. Nadiradze, “Relaxed schedulers can efficiently parallelize iterative algorithms,” in Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18, Egham, United Kingdom, 2018, pp. 377–386. ista: 'Alistarh D-A, Brown TA, Kopinsky J, Nadiradze G. 2018. Relaxed schedulers can efficiently parallelize iterative algorithms. Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18. PODC: Principles of Distributed Computing, 377–386.' mla: Alistarh, Dan-Adrian, et al. “Relaxed Schedulers Can Efficiently Parallelize Iterative Algorithms.” Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18, ACM, 2018, pp. 377–86, doi:10.1145/3212734.3212756. short: D.-A. Alistarh, T.A. Brown, J. Kopinsky, G. Nadiradze, in:, Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18, ACM, 2018, pp. 377–386. conference: end_date: 2018-07-27 location: Egham, United Kingdom name: 'PODC: Principles of Distributed Computing' start_date: 2018-07-23 date_created: 2019-02-13T10:03:25Z date_published: 2018-07-23T00:00:00Z date_updated: 2025-06-03T11:56:49Z day: '23' department: - _id: DaAl doi: 10.1145/3212734.3212756 external_id: arxiv: - '1808.04155' isi: - '000458186900048' isi: 1 language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/1808.04155 month: '07' oa: 1 oa_version: Preprint page: 377-386 publication: Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing - PODC '18 publication_identifier: isbn: - '9781450357951' publication_status: published publisher: ACM quality_controlled: '1' scopus_import: '1' status: public title: Relaxed schedulers can efficiently parallelize iterative algorithms type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 year: '2018' ... --- _id: '5964' abstract: - lang: eng text: A standard design pattern found in many concurrent data structures, such as hash tables or ordered containers, is an alternation of parallelizable sections that incur no data conflicts and critical sections that must run sequentially and are protected with locks. A lock can be viewed as a queue that arbitrates the order in which the critical sections are executed, and a natural question is whether we can use stochastic analysis to predict the resulting throughput. As a preliminary evidence to the affirmative, we describe a simple model that can be used to predict the throughput of coarse-grained lock-based algorithms. We show that our model works well for CLH lock, and we expect it to work for other popular lock designs such as TTAS, MCS, etc. article_processing_charge: No author: - first_name: Vitaly full_name: Aksenov, Vitaly last_name: Aksenov - first_name: Dan-Adrian full_name: Alistarh, Dan-Adrian id: 4A899BFC-F248-11E8-B48F-1D18A9856A87 last_name: Alistarh orcid: 0000-0003-3650-940X - first_name: Petr full_name: Kuznetsov, Petr last_name: Kuznetsov citation: ama: 'Aksenov V, Alistarh D-A, Kuznetsov P. Brief Announcement: Performance prediction for coarse-grained locking. In: Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18. ACM; 2018:411-413. doi:10.1145/3212734.3212785' apa: 'Aksenov, V., Alistarh, D.-A., & Kuznetsov, P. (2018). Brief Announcement: Performance prediction for coarse-grained locking. In Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18 (pp. 411–413). Egham, United Kingdom: ACM. https://doi.org/10.1145/3212734.3212785' chicago: 'Aksenov, Vitaly, Dan-Adrian Alistarh, and Petr Kuznetsov. “Brief Announcement: Performance Prediction for Coarse-Grained Locking.” In Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18, 411–13. ACM, 2018. https://doi.org/10.1145/3212734.3212785.' ieee: 'V. Aksenov, D.-A. Alistarh, and P. Kuznetsov, “Brief Announcement: Performance prediction for coarse-grained locking,” in Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18, Egham, United Kingdom, 2018, pp. 411–413.' ista: 'Aksenov V, Alistarh D-A, Kuznetsov P. 2018. Brief Announcement: Performance prediction for coarse-grained locking. Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18. PODC: Principles of Distributed Computing, 411–413.' mla: 'Aksenov, Vitaly, et al. “Brief Announcement: Performance Prediction for Coarse-Grained Locking.” Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18, ACM, 2018, pp. 411–13, doi:10.1145/3212734.3212785.' short: V. Aksenov, D.-A. Alistarh, P. Kuznetsov, in:, Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing  - PODC ’18, ACM, 2018, pp. 411–413. conference: end_date: 2018-07-27 location: Egham, United Kingdom name: 'PODC: Principles of Distributed Computing' start_date: 2018-07-23 date_created: 2019-02-13T10:08:19Z date_published: 2018-07-23T00:00:00Z date_updated: 2025-06-03T11:58:00Z day: '23' department: - _id: DaAl doi: 10.1145/3212734.3212785 external_id: isi: - '000458186900052' isi: 1 language: - iso: eng main_file_link: - open_access: '1' url: https://hal-univ-lyon3.archives-ouvertes.fr/INRIA/hal-01887733v1 month: '07' oa: 1 oa_version: Submitted Version page: 411-413 publication: Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing - PODC '18 publication_identifier: isbn: - '9781450357951' publication_status: published publisher: ACM quality_controlled: '1' scopus_import: '1' status: public title: 'Brief Announcement: Performance prediction for coarse-grained locking' type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 year: '2018' ... --- _id: '5965' abstract: - lang: eng text: Relaxed concurrent data structures have become increasingly popular, due to their scalability in graph processing and machine learning applications (\citeNguyen13, gonzalez2012powergraph ). Despite considerable interest, there exist families of natural, high performing randomized relaxed concurrent data structures, such as the popular MultiQueue~\citeMQ pattern for implementing relaxed priority queue data structures, for which no guarantees are known in the concurrent setting~\citeAKLN17. Our main contribution is in showing for the first time that, under a set of analytic assumptions, a family of relaxed concurrent data structures, including variants of MultiQueues, but also a new approximate counting algorithm we call the MultiCounter, provides strong probabilistic guarantees on the degree of relaxation with respect to the sequential specification, in arbitrary concurrent executions. We formalize these guarantees via a new correctness condition called distributional linearizability, tailored to concurrent implementations with randomized relaxations. Our result is based on a new analysis of an asynchronous variant of the classic power-of-two-choices load balancing algorithm, in which placement choices can be based on inconsistent, outdated information (this result may be of independent interest). We validate our results empirically, showing that the MultiCounter algorithm can implement scalable relaxed timestamps. article_processing_charge: No arxiv: 1 author: - first_name: Dan-Adrian full_name: Alistarh, Dan-Adrian id: 4A899BFC-F248-11E8-B48F-1D18A9856A87 last_name: Alistarh orcid: 0000-0003-3650-940X - first_name: Trevor A full_name: Brown, Trevor A id: 3569F0A0-F248-11E8-B48F-1D18A9856A87 last_name: Brown - first_name: Justin full_name: Kopinsky, Justin last_name: Kopinsky - first_name: Jerry Z. full_name: Li, Jerry Z. last_name: Li - first_name: Giorgi full_name: Nadiradze, Giorgi last_name: Nadiradze citation: ama: 'Alistarh D-A, Brown TA, Kopinsky J, Li JZ, Nadiradze G. Distributionally linearizable data structures. In: Proceedings of the 30th on Symposium on Parallelism in Algorithms and Architectures  - SPAA ’18. ACM; 2018:133-142. doi:10.1145/3210377.3210411' apa: 'Alistarh, D.-A., Brown, T. A., Kopinsky, J., Li, J. Z., & Nadiradze, G. (2018). Distributionally linearizable data structures. In Proceedings of the 30th on Symposium on Parallelism in Algorithms and Architectures  - SPAA ’18 (pp. 133–142). Vienna, Austria: ACM. https://doi.org/10.1145/3210377.3210411' chicago: Alistarh, Dan-Adrian, Trevor A Brown, Justin Kopinsky, Jerry Z. Li, and Giorgi Nadiradze. “Distributionally Linearizable Data Structures.” In Proceedings of the 30th on Symposium on Parallelism in Algorithms and Architectures  - SPAA ’18, 133–42. ACM, 2018. https://doi.org/10.1145/3210377.3210411. ieee: D.-A. Alistarh, T. A. Brown, J. Kopinsky, J. Z. Li, and G. Nadiradze, “Distributionally linearizable data structures,” in Proceedings of the 30th on Symposium on Parallelism in Algorithms and Architectures  - SPAA ’18, Vienna, Austria, 2018, pp. 133–142. ista: 'Alistarh D-A, Brown TA, Kopinsky J, Li JZ, Nadiradze G. 2018. Distributionally linearizable data structures. Proceedings of the 30th on Symposium on Parallelism in Algorithms and Architectures  - SPAA ’18. SPAA: Symposium on Parallelism in Algorithms and Architectures, 133–142.' mla: Alistarh, Dan-Adrian, et al. “Distributionally Linearizable Data Structures.” Proceedings of the 30th on Symposium on Parallelism in Algorithms and Architectures  - SPAA ’18, ACM, 2018, pp. 133–42, doi:10.1145/3210377.3210411. short: D.-A. Alistarh, T.A. Brown, J. Kopinsky, J.Z. Li, G. Nadiradze, in:, Proceedings of the 30th on Symposium on Parallelism in Algorithms and Architectures  - SPAA ’18, ACM, 2018, pp. 133–142. conference: end_date: 2018-07-18 location: Vienna, Austria name: 'SPAA: Symposium on Parallelism in Algorithms and Architectures' start_date: 2018-07-16 date_created: 2019-02-13T10:17:19Z date_published: 2018-07-16T00:00:00Z date_updated: 2026-04-08T07:00:45Z day: '16' department: - _id: DaAl doi: 10.1145/3210377.3210411 external_id: arxiv: - '1804.01018' isi: - '000545269600016' isi: 1 language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/1804.01018 month: '07' oa: 1 oa_version: Preprint page: 133-142 publication: Proceedings of the 30th on Symposium on Parallelism in Algorithms and Architectures - SPAA '18 publication_identifier: isbn: - '9781450357999' publication_status: published publisher: ACM quality_controlled: '1' related_material: record: - id: '10429' relation: dissertation_contains status: public scopus_import: '1' status: public title: Distributionally linearizable data structures type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 year: '2018' ... --- _id: '5966' abstract: - lang: eng text: 'The transactional conflict problem arises in transactional systems whenever two or more concurrent transactions clash on a data item. While the standard solution to such conflicts is to immediately abort one of the transactions, some practical systems consider the alternative of delaying conflict resolution for a short interval, which may allow one of the transactions to commit. The challenge in the transactional conflict problem is to choose the optimal length of this delay interval so as to minimize the overall running time penalty for the conflicting transactions. In this paper, we propose a family of optimal online algorithms for the transactional conflict problem. Specifically, we consider variants of this problem which arise in different implementations of transactional systems, namely "requestor wins'''' and "requestor aborts'''' implementations: in the former, the recipient of a coherence request is aborted, whereas in the latter, it is the requestor which has to abort. Both strategies are implemented by real systems. We show that the requestor aborts case can be reduced to a classic instance of the ski rental problem, while the requestor wins case leads to a new version of this classical problem, for which we derive optimal deterministic and randomized algorithms. Moreover, we prove that, under a simplified adversarial model, our algorithms are constant-competitive with the offline optimum in terms of throughput. We validate our algorithmic results empirically through a hardware simulation of hardware transactional memory (HTM), showing that our algorithms can lead to non-trivial performance improvements for classic concurrent data structures.' article_processing_charge: No arxiv: 1 author: - first_name: Dan-Adrian full_name: Alistarh, Dan-Adrian id: 4A899BFC-F248-11E8-B48F-1D18A9856A87 last_name: Alistarh orcid: 0000-0003-3650-940X - first_name: Syed Kamran full_name: Haider, Syed Kamran last_name: Haider - first_name: Raphael full_name: Kübler, Raphael last_name: Kübler - first_name: Giorgi full_name: Nadiradze, Giorgi last_name: Nadiradze citation: ama: 'Alistarh D-A, Haider SK, Kübler R, Nadiradze G. The transactional conflict problem. In: Proceedings of the 30th on Symposium on Parallelism in Algorithms and Architectures  - SPAA ’18. ACM; 2018:383-392. doi:10.1145/3210377.3210406' apa: 'Alistarh, D.-A., Haider, S. K., Kübler, R., & Nadiradze, G. (2018). The transactional conflict problem. In Proceedings of the 30th on Symposium on Parallelism in Algorithms and Architectures  - SPAA ’18 (pp. 383–392). Vienna, Austria: ACM. https://doi.org/10.1145/3210377.3210406' chicago: Alistarh, Dan-Adrian, Syed Kamran Haider, Raphael Kübler, and Giorgi Nadiradze. “The Transactional Conflict Problem.” In Proceedings of the 30th on Symposium on Parallelism in Algorithms and Architectures  - SPAA ’18, 383–92. ACM, 2018. https://doi.org/10.1145/3210377.3210406. ieee: D.-A. Alistarh, S. K. Haider, R. Kübler, and G. Nadiradze, “The transactional conflict problem,” in Proceedings of the 30th on Symposium on Parallelism in Algorithms and Architectures  - SPAA ’18, Vienna, Austria, 2018, pp. 383–392. ista: 'Alistarh D-A, Haider SK, Kübler R, Nadiradze G. 2018. The transactional conflict problem. Proceedings of the 30th on Symposium on Parallelism in Algorithms and Architectures  - SPAA ’18. SPAA: Symposium on Parallelism in Algorithms and Architectures, 383–392.' mla: Alistarh, Dan-Adrian, et al. “The Transactional Conflict Problem.” Proceedings of the 30th on Symposium on Parallelism in Algorithms and Architectures  - SPAA ’18, ACM, 2018, pp. 383–92, doi:10.1145/3210377.3210406. short: D.-A. Alistarh, S.K. Haider, R. Kübler, G. Nadiradze, in:, Proceedings of the 30th on Symposium on Parallelism in Algorithms and Architectures  - SPAA ’18, ACM, 2018, pp. 383–392. conference: end_date: 2018-07-18 location: Vienna, Austria name: 'SPAA: Symposium on Parallelism in Algorithms and Architectures' start_date: 2018-07-16 date_created: 2019-02-13T10:26:07Z date_published: 2018-07-16T00:00:00Z date_updated: 2025-06-03T11:58:22Z day: '16' department: - _id: DaAl doi: 10.1145/3210377.3210406 external_id: arxiv: - '1804.00947' isi: - '000545269600046' isi: 1 language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/1804.00947 month: '07' oa: 1 oa_version: Preprint page: 383-392 publication: Proceedings of the 30th on Symposium on Parallelism in Algorithms and Architectures - SPAA '18 publication_identifier: isbn: - '9781450357999' publication_status: published publisher: ACM quality_controlled: '1' scopus_import: '1' status: public title: The transactional conflict problem type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 year: '2018' ... --- _id: '6001' abstract: - lang: eng text: "The concurrent memory reclamation problem is that of devising a way for a deallocating thread to verify that no other concurrent threads hold references to a memory block being deallocated. To date, in the absence of automatic garbage collection, there is no satisfactory solution to this problem; existing tracking methods like hazard pointers, reference counters, or epoch-based techniques like RCU are either prohibitively expensive or require significant programming expertise to the extent that implementing them efficiently can be worthy of a publication. None of the existing techniques are automatic or even semi-automated.\r\nIn this article, we take a new approach to concurrent memory reclamation. Instead of manually tracking access to memory locations as done in techniques like hazard pointers, or restricting shared accesses to specific epoch boundaries as in RCU, our algorithm, called ThreadScan, leverages operating system signaling to automatically detect which memory locations are being accessed by concurrent threads.\r\nInitial empirical evidence shows that ThreadScan scales surprisingly well and requires negligible programming effort beyond the standard use of Malloc and Free." article_number: '18' author: - first_name: Dan-Adrian full_name: Alistarh, Dan-Adrian id: 4A899BFC-F248-11E8-B48F-1D18A9856A87 last_name: Alistarh orcid: 0000-0003-3650-940X - first_name: William full_name: Leiserson, William last_name: Leiserson - first_name: Alexander full_name: Matveev, Alexander last_name: Matveev - first_name: Nir full_name: Shavit, Nir last_name: Shavit citation: ama: 'Alistarh D-A, Leiserson W, Matveev A, Shavit N. ThreadScan: Automatic and scalable memory reclamation. ACM Transactions on Parallel Computing. 2018;4(4). doi:10.1145/3201897' apa: 'Alistarh, D.-A., Leiserson, W., Matveev, A., & Shavit, N. (2018). ThreadScan: Automatic and scalable memory reclamation. ACM Transactions on Parallel Computing. Association for Computing Machinery. https://doi.org/10.1145/3201897' chicago: 'Alistarh, Dan-Adrian, William Leiserson, Alexander Matveev, and Nir Shavit. “ThreadScan: Automatic and Scalable Memory Reclamation.” ACM Transactions on Parallel Computing. Association for Computing Machinery, 2018. https://doi.org/10.1145/3201897.' ieee: 'D.-A. Alistarh, W. Leiserson, A. Matveev, and N. Shavit, “ThreadScan: Automatic and scalable memory reclamation,” ACM Transactions on Parallel Computing, vol. 4, no. 4. Association for Computing Machinery, 2018.' ista: 'Alistarh D-A, Leiserson W, Matveev A, Shavit N. 2018. ThreadScan: Automatic and scalable memory reclamation. ACM Transactions on Parallel Computing. 4(4), 18.' mla: 'Alistarh, Dan-Adrian, et al. “ThreadScan: Automatic and Scalable Memory Reclamation.” ACM Transactions on Parallel Computing, vol. 4, no. 4, 18, Association for Computing Machinery, 2018, doi:10.1145/3201897.' short: D.-A. Alistarh, W. Leiserson, A. Matveev, N. Shavit, ACM Transactions on Parallel Computing 4 (2018). date_created: 2019-02-14T13:24:11Z date_published: 2018-09-01T00:00:00Z date_updated: 2023-02-23T13:17:54Z day: '01' department: - _id: DaAl doi: 10.1145/3201897 intvolume: ' 4' issue: '4' language: - iso: eng month: '09' oa_version: None publication: ACM Transactions on Parallel Computing publication_identifier: issn: - 2329-4949 publication_status: published publisher: Association for Computing Machinery quality_controlled: '1' related_material: record: - id: '779' relation: earlier_version status: public scopus_import: 1 status: public title: 'ThreadScan: Automatic and scalable memory reclamation' type: journal_article user_id: 3E5EF7F0-F248-11E8-B48F-1D18A9856A87 volume: 4 year: '2018' ... --- _id: '6031' abstract: - lang: eng text: We introduce Clover, a new library for efficient computation using low-precision data, providing mathematical routines required by fundamental methods in optimization and sparse recovery. Our library faithfully implements variants of stochastic quantization that guarantee convergence at low precision, and supports data formats from 4-bit quantized to 32-bit IEEE-754 on current Intel processors. In particular, we show that 4-bit can be implemented efficiently using Intel AVX despite the lack of native support for this data format. Experimental results with dot product, matrix-vector multiplication (MVM), gradient descent (GD), and iterative hard thresholding (IHT) demonstrate that the attainable speedups are in many cases close to linear with respect to the reduction of precision due to reduced data movement. Finally, for GD and IHT, we show examples of absolute speedup achieved by 4-bit versus 32-bit, by iterating until a given target error is achieved. article_number: '8598402' article_processing_charge: No author: - first_name: Alen full_name: Stojanov, Alen last_name: Stojanov - first_name: Tyler Michael full_name: Smith, Tyler Michael last_name: Smith - first_name: Dan-Adrian full_name: Alistarh, Dan-Adrian id: 4A899BFC-F248-11E8-B48F-1D18A9856A87 last_name: Alistarh orcid: 0000-0003-3650-940X - first_name: Markus full_name: Puschel, Markus last_name: Puschel citation: ama: 'Stojanov A, Smith TM, Alistarh D-A, Puschel M. Fast quantized arithmetic on x86: Trading compute for data movement. In: 2018 IEEE International Workshop on Signal Processing Systems. Vol 2018-October. IEEE; 2018. doi:10.1109/SiPS.2018.8598402' apa: 'Stojanov, A., Smith, T. M., Alistarh, D.-A., & Puschel, M. (2018). Fast quantized arithmetic on x86: Trading compute for data movement. In 2018 IEEE International Workshop on Signal Processing Systems (Vol. 2018–October). Cape Town, South Africa: IEEE. https://doi.org/10.1109/SiPS.2018.8598402' chicago: 'Stojanov, Alen, Tyler Michael Smith, Dan-Adrian Alistarh, and Markus Puschel. “Fast Quantized Arithmetic on X86: Trading Compute for Data Movement.” In 2018 IEEE International Workshop on Signal Processing Systems, Vol. 2018–October. IEEE, 2018. https://doi.org/10.1109/SiPS.2018.8598402.' ieee: 'A. Stojanov, T. M. Smith, D.-A. Alistarh, and M. Puschel, “Fast quantized arithmetic on x86: Trading compute for data movement,” in 2018 IEEE International Workshop on Signal Processing Systems, Cape Town, South Africa, 2018, vol. 2018–October.' ista: 'Stojanov A, Smith TM, Alistarh D-A, Puschel M. 2018. Fast quantized arithmetic on x86: Trading compute for data movement. 2018 IEEE International Workshop on Signal Processing Systems. SiPS: Workshop on Signal Processing Systems vol. 2018–October, 8598402.' mla: 'Stojanov, Alen, et al. “Fast Quantized Arithmetic on X86: Trading Compute for Data Movement.” 2018 IEEE International Workshop on Signal Processing Systems, vol. 2018–October, 8598402, IEEE, 2018, doi:10.1109/SiPS.2018.8598402.' short: A. Stojanov, T.M. Smith, D.-A. Alistarh, M. Puschel, in:, 2018 IEEE International Workshop on Signal Processing Systems, IEEE, 2018. conference: end_date: 2018-10-24 location: Cape Town, South Africa name: 'SiPS: Workshop on Signal Processing Systems' start_date: 2018-10-21 date_created: 2019-02-17T22:59:25Z date_published: 2018-12-31T00:00:00Z date_updated: 2023-09-19T14:41:51Z day: '31' department: - _id: DaAl doi: 10.1109/SiPS.2018.8598402 external_id: isi: - '000465106800060' isi: 1 language: - iso: eng month: '12' oa_version: None publication: 2018 IEEE International Workshop on Signal Processing Systems publication_status: published publisher: IEEE quality_controlled: '1' scopus_import: '1' status: public title: 'Fast quantized arithmetic on x86: Trading compute for data movement' type: conference user_id: c635000d-4b10-11ee-a964-aac5a93f6ac1 volume: 2018-October year: '2018' ... --- _id: '76' abstract: - lang: eng text: 'Consider a fully-connected synchronous distributed system consisting of n nodes, where up to f nodes may be faulty and every node starts in an arbitrary initial state. In the synchronous C-counting problem, all nodes need to eventually agree on a counter that is increased by one modulo C in each round for given C>1. In the self-stabilising firing squad problem, the task is to eventually guarantee that all non-faulty nodes have simultaneous responses to external inputs: if a subset of the correct nodes receive an external “go” signal as input, then all correct nodes should agree on a round (in the not-too-distant future) in which to jointly output a “fire” signal. Moreover, no node should generate a “fire” signal without some correct node having previously received a “go” signal as input. We present a framework reducing both tasks to binary consensus at very small cost. For example, we obtain a deterministic algorithm for self-stabilising Byzantine firing squads with optimal resilience f<n/3, asymptotically optimal stabilisation and response time O(f), and message size O(log f). As our framework does not restrict the type of consensus routines used, we also obtain efficient randomised solutions.' article_processing_charge: Yes (via OA deal) author: - first_name: Christoph full_name: Lenzen, Christoph last_name: Lenzen - first_name: Joel full_name: Rybicki, Joel id: 334EFD2E-F248-11E8-B48F-1D18A9856A87 last_name: Rybicki orcid: 0000-0002-6432-6646 citation: ama: Lenzen C, Rybicki J. Near-optimal self-stabilising counting and firing squads. Distributed Computing. 2018. doi:10.1007/s00446-018-0342-6 apa: Lenzen, C., & Rybicki, J. (2018). Near-optimal self-stabilising counting and firing squads. Distributed Computing. Springer. https://doi.org/10.1007/s00446-018-0342-6 chicago: Lenzen, Christoph, and Joel Rybicki. “Near-Optimal Self-Stabilising Counting and Firing Squads.” Distributed Computing. Springer, 2018. https://doi.org/10.1007/s00446-018-0342-6. ieee: C. Lenzen and J. Rybicki, “Near-optimal self-stabilising counting and firing squads,” Distributed Computing. Springer, 2018. ista: Lenzen C, Rybicki J. 2018. Near-optimal self-stabilising counting and firing squads. Distributed Computing. mla: Lenzen, Christoph, and Joel Rybicki. “Near-Optimal Self-Stabilising Counting and Firing Squads.” Distributed Computing, Springer, 2018, doi:10.1007/s00446-018-0342-6. short: C. Lenzen, J. Rybicki, Distributed Computing (2018). corr_author: '1' date_created: 2018-12-11T11:44:30Z date_published: 2018-09-12T00:00:00Z date_updated: 2025-04-15T06:53:15Z day: '12' ddc: - '000' department: - _id: DaAl doi: 10.1007/s00446-018-0342-6 external_id: isi: - '000475627800005' file: - access_level: open_access checksum: 872db70bba9b401500abe3c6ae2f1a61 content_type: application/pdf creator: dernst date_created: 2018-12-17T14:21:22Z date_updated: 2020-07-14T12:48:01Z file_id: '5711' file_name: 2018_DistributedComputing_Lenzen.pdf file_size: 799337 relation: main_file file_date_updated: 2020-07-14T12:48:01Z has_accepted_license: '1' isi: 1 language: - iso: eng month: '09' oa: 1 oa_version: Published Version project: - _id: B67AFEDC-15C9-11EA-A837-991A96BB2854 name: IST Austria Open Access Fund publication: Distributed Computing publication_status: published publisher: Springer publist_id: '7978' quality_controlled: '1' scopus_import: '1' status: public title: Near-optimal self-stabilising counting and firing squads tmp: image: /images/cc_by.png legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0) short: CC BY (4.0) type: journal_article user_id: c635000d-4b10-11ee-a964-aac5a93f6ac1 year: '2018' ... --- _id: '7812' abstract: - lang: eng text: Deep neural networks (DNNs) continue to make significant advances, solving tasks from image classification to translation or reinforcement learning. One aspect of the field receiving considerable attention is efficiently executing deep models in resource-constrained environments, such as mobile or embedded devices. This paper focuses on this problem, and proposes two new compression methods, which jointly leverage weight quantization and distillation of larger teacher networks into smaller student networks. The first method we propose is called quantized distillation and leverages distillation during the training process, by incorporating distillation loss, expressed with respect to the teacher, into the training of a student network whose weights are quantized to a limited set of levels. The second method, differentiable quantization, optimizes the location of quantization points through stochastic gradient descent, to better fit the behavior of the teacher model. We validate both methods through experiments on convolutional and recurrent architectures. We show that quantized shallow students can reach similar accuracy levels to full-precision teacher models, while providing order of magnitude compression, and inference speedup that is linear in the depth reduction. In sum, our results enable DNNs for resource-constrained environments to leverage architecture and accuracy advances developed on more powerful devices. acknowledgement: "We would like to thank Ce Zhang (ETH Zurich), Hantian Zhang (ETH Zurich) and Martin Jaggi ´\r\n(EPFL) for their support with experiments and valuable feedback.\r\n" article_processing_charge: No arxiv: 1 author: - first_name: Antonio full_name: Polino, Antonio last_name: Polino - first_name: Razvan full_name: Pascanu, Razvan last_name: Pascanu - first_name: Dan-Adrian full_name: Alistarh, Dan-Adrian id: 4A899BFC-F248-11E8-B48F-1D18A9856A87 last_name: Alistarh orcid: 0000-0003-3650-940X citation: ama: 'Polino A, Pascanu R, Alistarh D-A. Model compression via distillation and quantization. In: 6th International Conference on Learning Representations. ; 2018.' apa: Polino, A., Pascanu, R., & Alistarh, D.-A. (2018). Model compression via distillation and quantization. In 6th International Conference on Learning Representations. Vancouver, Canada. chicago: Polino, Antonio, Razvan Pascanu, and Dan-Adrian Alistarh. “Model Compression via Distillation and Quantization.” In 6th International Conference on Learning Representations, 2018. ieee: A. Polino, R. Pascanu, and D.-A. Alistarh, “Model compression via distillation and quantization,” in 6th International Conference on Learning Representations, Vancouver, Canada, 2018. ista: 'Polino A, Pascanu R, Alistarh D-A. 2018. Model compression via distillation and quantization. 6th International Conference on Learning Representations. ICLR: International Conference on Learning Representations.' mla: Polino, Antonio, et al. “Model Compression via Distillation and Quantization.” 6th International Conference on Learning Representations, 2018. short: A. Polino, R. Pascanu, D.-A. Alistarh, in:, 6th International Conference on Learning Representations, 2018. conference: end_date: 2018-05-03 location: Vancouver, Canada name: 'ICLR: International Conference on Learning Representations' start_date: 2018-04-30 date_created: 2020-05-10T22:00:51Z date_published: 2018-05-01T00:00:00Z date_updated: 2025-06-30T10:04:44Z day: '01' ddc: - '000' department: - _id: DaAl external_id: arxiv: - '1802.05668' file: - access_level: open_access checksum: a4336c167978e81891970e4e4517a8c3 content_type: application/pdf creator: dernst date_created: 2020-05-26T13:02:00Z date_updated: 2020-07-14T12:48:03Z file_id: '7894' file_name: 2018_ICLR_Polino.pdf file_size: 308339 relation: main_file file_date_updated: 2020-07-14T12:48:03Z has_accepted_license: '1' language: - iso: eng month: '05' oa: 1 oa_version: Published Version publication: 6th International Conference on Learning Representations publication_status: published quality_controlled: '1' scopus_import: '1' status: public title: Model compression via distillation and quantization type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 year: '2018' ... --- _id: '85' abstract: - lang: eng text: Concurrent accesses to shared data structures must be synchronized to avoid data races. Coarse-grained synchronization, which locks the entire data structure, is easy to implement but does not scale. Fine-grained synchronization can scale well, but can be hard to reason about. Hand-over-hand locking, in which operations are pipelined as they traverse the data structure, combines fine-grained synchronization with ease of use. However, the traditional implementation suffers from inherent overheads. This paper introduces snapshot-based synchronization (SBS), a novel hand-over-hand locking mechanism. SBS decouples the synchronization state from the data, significantly improving cache utilization. Further, it relies on guarantees provided by pipelining to minimize synchronization that requires cross-thread communication. Snapshot-based synchronization thus scales much better than traditional hand-over-hand locking, while maintaining the same ease of use. acknowledgement: Trevor Brown was supported in part by the ISF (grants 2005/17 & 1749/14) and by a NSERC post-doctoral fellowship. alternative_title: - LNCS article_processing_charge: No author: - first_name: Eran full_name: Gilad, Eran last_name: Gilad - first_name: Trevor A full_name: Brown, Trevor A id: 3569F0A0-F248-11E8-B48F-1D18A9856A87 last_name: Brown - first_name: Mark full_name: Oskin, Mark last_name: Oskin - first_name: Yoav full_name: Etsion, Yoav last_name: Etsion citation: ama: 'Gilad E, Brown TA, Oskin M, Etsion Y. Snapshot based synchronization: A fast replacement for Hand-over-Hand locking. In: Vol 11014. Springer; 2018:465-479. doi:10.1007/978-3-319-96983-1_33' apa: 'Gilad, E., Brown, T. A., Oskin, M., & Etsion, Y. (2018). Snapshot based synchronization: A fast replacement for Hand-over-Hand locking (Vol. 11014, pp. 465–479). Presented at the Euro-Par: European Conference on Parallel Processing, Turin, Italy: Springer. https://doi.org/10.1007/978-3-319-96983-1_33' chicago: 'Gilad, Eran, Trevor A Brown, Mark Oskin, and Yoav Etsion. “Snapshot Based Synchronization: A Fast Replacement for Hand-over-Hand Locking,” 11014:465–79. Springer, 2018. https://doi.org/10.1007/978-3-319-96983-1_33.' ieee: 'E. Gilad, T. A. Brown, M. Oskin, and Y. Etsion, “Snapshot based synchronization: A fast replacement for Hand-over-Hand locking,” presented at the Euro-Par: European Conference on Parallel Processing, Turin, Italy, 2018, vol. 11014, pp. 465–479.' ista: 'Gilad E, Brown TA, Oskin M, Etsion Y. 2018. Snapshot based synchronization: A fast replacement for Hand-over-Hand locking. Euro-Par: European Conference on Parallel Processing, LNCS, vol. 11014, 465–479.' mla: 'Gilad, Eran, et al. Snapshot Based Synchronization: A Fast Replacement for Hand-over-Hand Locking. Vol. 11014, Springer, 2018, pp. 465–79, doi:10.1007/978-3-319-96983-1_33.' short: E. Gilad, T.A. Brown, M. Oskin, Y. Etsion, in:, Springer, 2018, pp. 465–479. conference: end_date: 2018-08-31 location: Turin, Italy name: 'Euro-Par: European Conference on Parallel Processing' start_date: 2018-08-27 date_created: 2018-12-11T11:44:33Z date_published: 2018-08-01T00:00:00Z date_updated: 2026-04-16T09:53:41Z day: '01' ddc: - '000' department: - _id: DaAl doi: 10.1007/978-3-319-96983-1_33 external_id: isi: - '000851042300031' file: - access_level: open_access checksum: 13a3f250be8878405e791b53c19722ad content_type: application/pdf creator: dernst date_created: 2019-02-12T07:40:40Z date_updated: 2020-07-14T12:48:14Z file_id: '5954' file_name: 2018_Brown.pdf file_size: 665372 relation: main_file file_date_updated: 2020-07-14T12:48:14Z has_accepted_license: '1' intvolume: ' 11014' isi: 1 language: - iso: eng month: '08' oa: 1 oa_version: Preprint page: 465 - 479 project: - _id: 26450934-B435-11E9-9278-68D0E5697425 name: NSERC Postdoctoral fellowship publication_identifier: issn: - 0302-9743 publication_status: published publisher: Springer publist_id: '7969' quality_controlled: '1' scopus_import: '1' status: public title: 'Snapshot based synchronization: A fast replacement for Hand-over-Hand locking' type: conference user_id: ba8df636-2132-11f1-aed0-ed93e2281fdd volume: 11014 year: '2018' ...