---
res:
  bibo_abstract:
  - "Large Language Models (LLMs) have demonstrated the ability to tackle increasingly
    complex tasks through advanced reasoning, long-form content generation,\r\nand
    tool use. Solving these tasks often involves long inference-time computations.\r\nIn
    human problem solving, a common strategy to expedite work is collaboration: by\r\ndividing
    the problem into sub-tasks, exploring different strategies concurrently, etc.\r\nRecent
    research has shown that LLMs can also operate in parallel by implementing\r\nexplicit
    cooperation frameworks, such as voting mechanisms or the explicit creation of
    independent sub-tasks that can be executed in parallel. However, each of\r\nthese
    frameworks may not be suitable for all types of tasks, which can hinder their\r\napplicability.
    In this work, we propose a different design approach: we run LLM\r\n“workers”
    in parallel , allowing them to synchronize via a concurrently-updated\r\nattention
    cache and prompt these workers to decide how best to collaborate. Our\r\napproach
    allows the LLM instances to come up with their own collaboration strategy for
    the problem at hand, all the while “seeing” each other’s memory in the\r\nconcurrent
    KV cache. We implement this approach via Hogwild! Inference: a\r\nparallel LLM
    inference engine where multiple instances of the same LLM run in\r\nparallel with
    the same attention cache, with “instant” access to each other’s memory.1 Hogwild!
    Inference takes advantage of Rotary Position Embeddings (RoPE)\r\nto avoid recomputation
    while improving parallel hardware utilization. We find that\r\nmodern reasoning-capable
    LLMs can perform inference with shared Key-Value\r\ncache out of the box, without
    additional fine-tuning.@eng"
  bibo_authorlist:
  - foaf_Person:
      foaf_givenName: Gleb
      foaf_name: Rodionov, Gleb
      foaf_surname: Rodionov
  - foaf_Person:
      foaf_givenName: Roman
      foaf_name: Garipov, Roman
      foaf_surname: Garipov
  - foaf_Person:
      foaf_givenName: Alina
      foaf_name: Shutova, Alina
      foaf_surname: Shutova
  - foaf_Person:
      foaf_givenName: George
      foaf_name: Yakushev, George
      foaf_surname: Yakushev
  - foaf_Person:
      foaf_givenName: Erik
      foaf_name: Schultheis, Erik
      foaf_surname: Schultheis
      foaf_workInfoHomepage: http://www.librecat.org/personId=2786b299-e6b0-11f0-91da-9243fe3ef96b
  - foaf_Person:
      foaf_givenName: Vage
      foaf_name: Egiazarian, Vage
      foaf_surname: Egiazarian
      foaf_workInfoHomepage: http://www.librecat.org/personId=77451e76-92b2-11ef-a4d1-8dbaa06e16ad
  - foaf_Person:
      foaf_givenName: Anton
      foaf_name: Sinitsin, Anton
      foaf_surname: Sinitsin
  - foaf_Person:
      foaf_givenName: Denis
      foaf_name: Kuznedelev, Denis
      foaf_surname: Kuznedelev
  - foaf_Person:
      foaf_givenName: Dan-Adrian
      foaf_name: Alistarh, Dan-Adrian
      foaf_surname: Alistarh
      foaf_workInfoHomepage: http://www.librecat.org/personId=4A899BFC-F248-11E8-B48F-1D18A9856A87
    orcid: 0000-0003-3650-940X
  bibo_doi: 10.52202/085713-1551
  bibo_volume: 38
  dct_date: 2025^xs_gYear
  dct_isPartOf:
  - http://id.crossref.org/issn/1049-5258
  - http://id.crossref.org/issn/9798331338275
  dct_language: eng
  dct_publisher: Neural Information Processing Systems Foundation@
  dct_title: 'Hogwild! Inference: Parallel LLM generation via concurrent attention@'
...