arXiv：cs.AI· Zhixiong Zhao, Zukang Xu, Xing Hu, Lifeng Liu, Guangyu Sun, Dawei Yang·· 22 小时前
LoopQuant：面向循环 Transformer 的精准高效训练后量化框架
LoopQuant: Accurate and Efficient Post-Training Quantization for Looped Transformers
AI 导读
针对循环语言模型（LoopLMs）低比特部署难题，研究者提出循环感知的训练后量化框架 LoopQuant，包含 CLDC、ILPA、LCTC 三个核心组件。在 Ouro-2.6B 上，LoopQuant 在 W4A4 下性能超过所有 W4A8 基线，模型内存减少 65.3%，仅比朴素 INT4 多 3.4% 存储开销，解码速度较 BF16 提升 1.7 倍。代码已开源。
正文
Abstract:Looped Language Models (LoopLMs) improve parameter efficiency by recursively reusing shared Transformer modules, yet their repeated computation over evolving recurrent states poses new challenges for low-bit deployment. Existing post-training quantization (PTQ) methods, largely designed for feed-forward Transformers, fail to account for cross-loop activation dynamics, heterogeneous loop sensitivity, and recursive error propagation, leading to substantial performance degradation at low precision. To address these challenges, we propose LoopQuant, a loop-aware PTQ framework for accurate and efficient low-bit quantization of LoopLMs. LoopQuant is built on three core components: 1) Cross-Loop Distribution Canonicalization (CLDC) learns a shared quantization-friendly representation across recurrent states while preserving parameter sharing; 2) Interaction-Aware Loop-wise Precision Allocation (ILPA) allocates activation precision under a fixed bit budget by modeling task-level loop sensitivity and cross-loop interactions; and 3) Loop-Conditioned Task Calibration (LCTC) further aligns quantized inference with the low-bit recursive trajectory through lightweight task-space calibration. Extensive experiments across LoopLM architectures demonstrate that LoopQuant consistently outperforms existing low-bit PTQ methods. On Ouro-2.6B, LoopQuant under W4A4 surpasses all W4A8 baselines in performance, while reducing model memory by 65.3% with only 3.4% additional storage overhead over naive INT4 and achieving a 1.7 times decoding speedup over BF16. These results demonstrate a strong balance among accuracy, compression, and inference efficiency, providing a practical and deployment-friendly solution for low-bit LoopLM inference. The code is available at [this https URL].
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.16368 [cs.LG] |
| (or arXiv:2610.16368v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.16368 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhixiong Zhao [view email]
[v1]
Mon, 12 Oct 2026 08:20:04 UTC (22,875 KB)
来源：arXiv：cs.AI · arxiv.org