---
OA_place: publisher
OA_type: diamond
_id: '21932'
abstract:
- lang: eng
  text: We present LLMQ, an end-to-end CUDA/C++ implementation for medium-sized language-model
    training, e.g. 3B to 32B parameters, on affordable, commodity GPUs. These devices
    are characterized by low memory availability and slow communication compared to
    datacentre-grade GPUs. Consequently, we showcase a range of optimizations that
    target these bottlenecks, including activation checkpointing, offloading, and
    copy-engine based collectives. LLMQ is able to train or fine-tune a 7B model on
    a single 16GB mid-range gaming card, or a 32B model on a workstation equipped
    with 4 RTX 4090s. This is achieved while executing a standard 8-bit training pipeline,
    without additional algorithmic approximations, and maintaining FLOP utilization
    of around 50%. The efficiency of LLMQ rivals that of production-scale systems
    on much more expensive cloud-grade GPUs.
acknowledgement: "We would like to thank contacts at NVIDIA (Vartika Singh, Nina Carrejo,
  Kyla Wilkes, and Tijmen\r\nBlankevoort), HP (Curtis Burkhalter), and Datacrunch/Verda
  (Paul Chang and Antonio\r\nDominguez) for hardware support that was essential to
  this project. ES was supported in part\r\nby ERC Proof-of-Concept grant FastML."
alternative_title:
- PMLR
article_processing_charge: No
author:
- first_name: Erik
  full_name: Schultheis, Erik
  id: 2786b299-e6b0-11f0-91da-9243fe3ef96b
  last_name: Schultheis
- first_name: Dan-Adrian
  full_name: Alistarh, Dan-Adrian
  id: 4A899BFC-F248-11E8-B48F-1D18A9856A87
  last_name: Alistarh
  orcid: 0000-0003-3650-940X
citation:
  ama: 'Schultheis E, Alistarh D-A. LLMQ: Efficient lower-precision LLM training for
    consumer GPUs. In: <i>2nd Conference on Parsimony and Learning</i>. Vol 328. ML
    Research Press; 2026:265-284.'
  apa: 'Schultheis, E., &#38; Alistarh, D.-A. (2026). LLMQ: Efficient lower-precision
    LLM training for consumer GPUs. In <i>2nd Conference on Parsimony and Learning</i>
    (Vol. 328, pp. 265–284). Stanford, CA, United States: ML Research Press.'
  chicago: 'Schultheis, Erik, and Dan-Adrian Alistarh. “LLMQ: Efficient Lower-Precision
    LLM Training for Consumer GPUs.” In <i>2nd Conference on Parsimony and Learning</i>,
    328:265–84. ML Research Press, 2026.'
  ieee: 'E. Schultheis and D.-A. Alistarh, “LLMQ: Efficient lower-precision LLM training
    for consumer GPUs,” in <i>2nd Conference on Parsimony and Learning</i>, Stanford,
    CA, United States, 2026, vol. 328, pp. 265–284.'
  ista: 'Schultheis E, Alistarh D-A. 2026. LLMQ: Efficient lower-precision LLM training
    for consumer GPUs. 2nd Conference on Parsimony and Learning. CPAL: Conference
    on Parsimony and Learning, PMLR, vol. 328, 265–284.'
  mla: 'Schultheis, Erik, and Dan-Adrian Alistarh. “LLMQ: Efficient Lower-Precision
    LLM Training for Consumer GPUs.” <i>2nd Conference on Parsimony and Learning</i>,
    vol. 328, ML Research Press, 2026, pp. 265–84.'
  short: E. Schultheis, D.-A. Alistarh, in:, 2nd Conference on Parsimony and Learning,
    ML Research Press, 2026, pp. 265–284.
conference:
  end_date: 2025-03-27
  location: Stanford, CA, United States
  name: 'CPAL: Conference on Parsimony and Learning'
  start_date: 2025-03-24
corr_author: '1'
date_created: 2026-05-31T22:02:13Z
date_published: 2026-04-06T00:00:00Z
date_updated: 2026-06-03T05:53:30Z
day: '06'
ddc:
- '000'
department:
- _id: DaAl
file:
- access_level: open_access
  checksum: 72f9a87c70f1e2105ef64050ee5017e5
  content_type: application/pdf
  creator: dernst
  date_created: 2026-06-03T05:51:19Z
  date_updated: 2026-06-03T05:51:19Z
  file_id: '21942'
  file_name: 2026_CPAL_Schultheis.pdf
  file_size: 2099944
  relation: main_file
  success: 1
file_date_updated: 2026-06-03T05:51:19Z
has_accepted_license: '1'
intvolume: '       328'
language:
- iso: eng
month: '04'
oa: 1
oa_version: Published Version
page: 265-284
publication: 2nd Conference on Parsimony and Learning
publication_identifier:
  eissn:
  - 2640-3498
publication_status: published
publisher: ML Research Press
quality_controlled: '1'
related_material:
  link:
  - relation: software
    url: https://github.com/IST-DASLab/llmq
scopus_import: '1'
status: public
title: 'LLMQ: Efficient lower-precision LLM training for consumer GPUs'
tmp:
  image: /images/cc_by.png
  legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode
  name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)
  short: CC BY (4.0)
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 328
year: '2026'
...