---
res:
  bibo_abstract:
  - "We introduce AutoJudge, a method that accelerates large language model (LLM)
    inference with task-specific lossy speculative decoding. Instead of matching the
    original model output distribution token-by-token, we identify the generated tokens
    that affect the downstream quality of the response, relaxing the distribution
    match guarantee so that the \"unimportant\" tokens can be generated faster. Our
    approach relies on a semi‑greedy search algorithm to test which of the mismatches
    between target and draft models should be corrected to preserve quality and which
    ones may be skipped. We then train a lightweight classifier based on existing
    LLM embeddings to predict, at inference time, which mismatching tokens can be
    safely accepted without compromising the final answer quality. We evaluate AutoJudge
    with multiple draft/target model pairs on mathematical reasoning and programming
    benchmarks, achieving significant speedups at the cost of a minor accuracy reduction.
    Notably, on GSM8K with the Llama 3.1 70B target model, our approach achieves up
    to \r\n≈\r\n2\r\n×\r\n speedup \\textit{over speculative decoding} at the cost
    of a \r\n≤\r\n1\r\n%\r\n drop in accuracy. When applied to the LiveCodeBench benchmark,
    AutoJudge automatically detects programming-specific important tokens, accepting
    \r\n≥\r\n25\r\n tokens per speculation cycle at a\r\n \r\n2\r\n%\r\n drop in Pass@1.
    Our approach requires no human annotation and is easy to integrate with modern
    LLM inference frameworks.@eng"
  bibo_authorlist:
  - foaf_Person:
      foaf_givenName: Roman
      foaf_name: Garipov, Roman
      foaf_surname: Garipov
  - foaf_Person:
      foaf_givenName: Fedor
      foaf_name: Velikonivtsev, Fedor
      foaf_surname: Velikonivtsev
  - foaf_Person:
      foaf_givenName: Ivan
      foaf_name: Ermakov, Ivan
      foaf_surname: Ermakov
  - foaf_Person:
      foaf_givenName: Ruslan
      foaf_name: Svirschevski, Ruslan
      foaf_surname: Svirschevski
  - foaf_Person:
      foaf_givenName: Vage
      foaf_name: Egiazarian, Vage
      foaf_surname: Egiazarian
      foaf_workInfoHomepage: http://www.librecat.org/personId=77451e76-92b2-11ef-a4d1-8dbaa06e16ad
  - foaf_Person:
      foaf_givenName: Max
      foaf_name: Ryabinin, Max
      foaf_surname: Ryabinin
  bibo_doi: 10.52202/085713-3163
  bibo_volume: 38
  dct_date: 2025^xs_gYear
  dct_isPartOf:
  - http://id.crossref.org/issn/1049-5258
  - http://id.crossref.org/issn/9798331338275
  dct_language: eng
  dct_publisher: Neural Information Processing Systems Foundation@
  dct_title: 'AutoJudge: Judge decoding without manual annotation@'
...