On achieving scalability through relaxation

Nadiradze G. 2021. On achieving scalability through relaxation. IST Austria.

OA Thesis_Final_09_12_2021.pdf 2.37 MB

Thesis | PhD | Published | English
Series Title
IST Austria Thesis
The scalability of concurrent data structures and distributed algorithms strongly depends on reducing the contention for shared resources and the costs of synchronization and communication. We show how such cost reductions can be attained by relaxing the strict consistency conditions required by sequential implementations. In the first part of the thesis, we consider relaxation in the context of concurrent data structures. Specifically, in data structures such as priority queues, imposing strong semantics renders scalability impossible, since a correct implementation of the remove operation should return only the element with highest priority. Intuitively, attempting to invoke remove operations concurrently creates a race condition. This bottleneck can be circumvented by relaxing semantics of the affected data structure, thus allowing removal of the elements which are no longer required to have the highest priority. We prove that the randomized implementations of relaxed data structures provide provable guarantees on the priority of the removed elements even under concurrency. Additionally, we show that in some cases the relaxed data structures can be used to scale the classical algorithms which are usually implemented with the exact ones. In the second part, we study parallel variants of the stochastic gradient descent (SGD) algorithm, which distribute computation among the multiple processors, thus reducing the running time. Unfortunately, in order for standard parallel SGD to succeed, each processor has to maintain a local copy of the necessary model parameter, which is identical to the local copies of other processors; the overheads from this perfect consistency in terms of communication and synchronization can negate the speedup gained by distributing the computation. We show that the consistency conditions required by SGD can be relaxed, allowing the algorithm to be more flexible in terms of tolerating quantized communication, asynchrony, or even crash faults, while its convergence remains asymptotically the same.
Publishing Year
Date Published

Cite this

Nadiradze G. On achieving scalability through relaxation. 2021. doi:10.15479/at:ista:10429
Nadiradze, G. (2021). On achieving scalability through relaxation. IST Austria. https://doi.org/10.15479/at:ista:10429
Nadiradze, Giorgi. “On Achieving Scalability through Relaxation.” IST Austria, 2021. https://doi.org/10.15479/at:ista:10429.
G. Nadiradze, “On achieving scalability through relaxation,” IST Austria, 2021.
Nadiradze G. 2021. On achieving scalability through relaxation. IST Austria.
Nadiradze, Giorgi. On Achieving Scalability through Relaxation. IST Austria, 2021, doi:10.15479/at:ista:10429.
All files available under the following license(s):
Copyright Statement:
This Item is protected by copyright and/or related rights. [...]
Main File(s)
Access Level
OA Open Access
Date Uploaded
MD5 Checksum

Source File
Access Level
Restricted Closed Access
Date Uploaded
MD5 Checksum


Marked Publications

Open Data ISTA Research Explorer

Search this title in

Google Scholar