{"_id":"6558","title":"Byzantine stochastic gradient descent","month":"12","oa_version":"Published Version","type":"conference","citation":{"short":"D.-A. Alistarh, Z. Allen-Zhu, J. Li, in:, Advances in Neural Information Processing Systems, Neural Information Processing Systems Foundation, 2018, pp. 4613–4623.","apa":"Alistarh, D.-A., Allen-Zhu, Z., &#38; Li, J. (2018). Byzantine stochastic gradient descent. In <i>Advances in Neural Information Processing Systems</i> (Vol. 2018, pp. 4613–4623). Montreal, Canada: Neural Information Processing Systems Foundation.","ista":"Alistarh D-A, Allen-Zhu Z, Li J. 2018. Byzantine stochastic gradient descent. Advances in Neural Information Processing Systems. NeurIPS: Conference on Neural Information Processing Systems vol. 2018, 4613–4623.","ama":"Alistarh D-A, Allen-Zhu Z, Li J. Byzantine stochastic gradient descent. In: <i>Advances in Neural Information Processing Systems</i>. Vol 2018. Neural Information Processing Systems Foundation; 2018:4613-4623.","chicago":"Alistarh, Dan-Adrian, Zeyuan Allen-Zhu, and Jerry Li. “Byzantine Stochastic Gradient Descent.” In <i>Advances in Neural Information Processing Systems</i>, 2018:4613–23. Neural Information Processing Systems Foundation, 2018.","ieee":"D.-A. Alistarh, Z. Allen-Zhu, and J. Li, “Byzantine stochastic gradient descent,” in <i>Advances in Neural Information Processing Systems</i>, Montreal, Canada, 2018, vol. 2018, pp. 4613–4623.","mla":"Alistarh, Dan-Adrian, et al. “Byzantine Stochastic Gradient Descent.” <i>Advances in Neural Information Processing Systems</i>, vol. 2018, Neural Information Processing Systems Foundation, 2018, pp. 4613–23."},"publisher":"Neural Information Processing Systems Foundation","scopus_import":"1","oa":1,"conference":{"name":"NeurIPS: Conference on Neural Information Processing Systems","end_date":"2018-12-08","start_date":"2018-12-02","location":"Montreal, Canada"},"publication_status":"published","language":[{"iso":"eng"}],"author":[{"last_name":"Alistarh","id":"4A899BFC-F248-11E8-B48F-1D18A9856A87","full_name":"Alistarh, Dan-Adrian","orcid":"0000-0003-3650-940X","first_name":"Dan-Adrian"},{"first_name":"Zeyuan","full_name":"Allen-Zhu, Zeyuan","last_name":"Allen-Zhu"},{"full_name":"Li, Jerry","first_name":"Jerry","last_name":"Li"}],"page":"4613-4623","user_id":"c635000d-4b10-11ee-a964-aac5a93f6ac1","department":[{"_id":"DaAl"}],"publication":"Advances in Neural Information Processing Systems","date_created":"2019-06-13T08:22:37Z","year":"2018","article_processing_charge":"No","date_published":"2018-12-01T00:00:00Z","external_id":{"arxiv":["1803.08917"],"isi":["000461823304061"]},"volume":2018,"isi":1,"main_file_link":[{"url":"https://arxiv.org/abs/1803.08917","open_access":"1"}],"intvolume":"      2018","date_updated":"2023-09-19T15:12:45Z","abstract":[{"lang":"eng","text":"This paper studies the problem of distributed stochastic optimization in an adversarial setting where, out of m machines which allegedly compute stochastic gradients every iteration, an α-fraction are Byzantine, and may behave adversarially. Our main result is a variant of stochastic gradient descent (SGD) which finds ε-approximate minimizers of convex functions in T=O~(1/ε²m+α²/ε²) iterations. In contrast, traditional mini-batch SGD needs T=O(1/ε²m) iterations, but cannot tolerate Byzantine failures. Further, we provide a lower bound showing that, up to logarithmic factors, our algorithm is information-theoretically optimal both in terms of sample complexity and time complexity."}],"status":"public","day":"01","quality_controlled":"1"}