---
res:
  bibo_abstract:
  - Distributed optimization is the standard way of speeding up machine learning training,
    and most of the research in the area focuses on distributed first-order, gradient-based
    methods. Yet, there are settings where some computationally-bounded nodes may
    not be able to implement first-order, gradient-based optimization, while they
    could still contribute to joint optimization tasks. In this paper, we initiate
    the study of hybrid decentralized optimization, studying settings where nodes
    with zeroth-order and first-order optimization capabilities co-exist in a distributed
    system, and attempt to jointly solve an optimization task over some data distribution.
    We essentially show that, under reasonable parameter settings, such a system can
    not only withstand noisier zeroth-order agents but can even benefit from integrating
    such agents into the optimization process, rather than ignoring their information.
    At the core of our approach is a new analysis of distributed optimization with
    noisy and possibly-biased gradient estimators, which may be of independent interest.
    Our results hold for both convex and non-convex objectives. Experimental results
    on standard optimization tasks confirm our analysis, showing that hybrid first-zeroth
    order optimization can be practical, even when training deep neural networks.@eng
  bibo_authorlist:
  - foaf_Person:
      foaf_givenName: Shayan
      foaf_name: Talaei, Shayan
      foaf_surname: Talaei
  - foaf_Person:
      foaf_givenName: Matin
      foaf_name: Ansaripour, Matin
      foaf_surname: Ansaripour
  - foaf_Person:
      foaf_givenName: Giorgi
      foaf_name: Nadiradze, Giorgi
      foaf_surname: Nadiradze
      foaf_workInfoHomepage: http://www.librecat.org/personId=3279A00C-F248-11E8-B48F-1D18A9856A87
    orcid: 0000-0001-5634-0731
  - foaf_Person:
      foaf_givenName: Dan-Adrian
      foaf_name: Alistarh, Dan-Adrian
      foaf_surname: Alistarh
      foaf_workInfoHomepage: http://www.librecat.org/personId=4A899BFC-F248-11E8-B48F-1D18A9856A87
    orcid: 0000-0003-3650-940X
  bibo_doi: 10.1609/aaai.v39i19.34290
  bibo_issue: '19'
  bibo_volume: 39
  dct_date: 2025^xs_gYear
  dct_isPartOf:
  - http://id.crossref.org/issn/2159-5399
  - http://id.crossref.org/issn/2374-3468
  dct_language: eng
  dct_publisher: Association for the Advancement of Artificial Intelligence@
  dct_title: 'Hybrid decentralized optimization: Leveraging both first- and zeroth-order
    optimizers for faster convergence@'
...