LLMQ: Efficient lower-precision LLM training for consumer GPUs

conference paper LLMQ: Efficient lower-precision LLM training for consumer GPUs PMLR published yes Erik Schultheis author 2786b299-e6b0-11f0-91da-9243fe3ef96b Dan-Adrian Alistarh author 4A899BFC-F248-11E8-B48F-1D18A9856A870000-0003-3650-940X DaAl department CPAL: Conference on Parsimony and Learning We present LLMQ, an end-to-end CUDA/C++ implementation for medium-sized language-model training, e.g. 3B to 32B parameters, on affordable, commodity GPUs. These devices are characterized by low memory availability and slow communication compared to datacentre-grade GPUs. Consequently, we showcase a range of optimizations that target these bottlenecks, including activation checkpointing, offloading, and copy-engine based collectives. LLMQ is able to train or fine-tune a 7B model on a single 16GB mid-range gaming card, or a 32B model on a workstation equipped with 4 RTX 4090s. This is achieved while executing a standard 8-bit training pipeline, without additional algorithmic approximations, and maintaining FLOP utilization of around 50%. The efficiency of LLMQ rivals that of production-scale systems on much more expensive cloud-grade GPUs. https://research-explorer.ista.ac.at/download/21932/21942/2026_CPAL_Schultheis.pdf application/pdfno ML Research Press2026Stanford, CA, United States eng 2nd Conference on Parsimony and Learning 2640-3498 328265-284 https://github.com/IST-DASLab/llmq Schultheis, E., & Alistarh, D.-A. (2026). LLMQ: Efficient lower-precision LLM training for consumer GPUs. In 2nd Conference on Parsimony and Learning (Vol. 328, pp. 265–284). Stanford, CA, United States: ML Research Press. Schultheis, Erik, and Dan-Adrian Alistarh. “LLMQ: Efficient Lower-Precision LLM Training for Consumer GPUs.” 2nd Conference on Parsimony and Learning, vol. 328, ML Research Press, 2026, pp. 265–84. Schultheis E, Alistarh D-A. LLMQ: Efficient lower-precision LLM training for consumer GPUs. In: 2nd Conference on Parsimony and Learning. Vol 328. ML Research Press; 2026:265-284. Schultheis, Erik, and Dan-Adrian Alistarh. “LLMQ: Efficient Lower-Precision LLM Training for Consumer GPUs.” In 2nd Conference on Parsimony and Learning, 328:265–84. ML Research Press, 2026. Schultheis E, Alistarh D-A. 2026. LLMQ: Efficient lower-precision LLM training for consumer GPUs. 2nd Conference on Parsimony and Learning. CPAL: Conference on Parsimony and Learning, PMLR, vol. 328, 265–284. E. Schultheis and D.-A. Alistarh, “LLMQ: Efficient lower-precision LLM training for consumer GPUs,” in 2nd Conference on Parsimony and Learning, Stanford, CA, United States, 2026, vol. 328, pp. 265–284. E. Schultheis, D.-A. Alistarh, in:, 2nd Conference on Parsimony and Learning, ML Research Press, 2026, pp. 265–284. 219322026-05-31T22:02:13Z2026-06-03T05:53:30Z