arXiv：cs.CL· Chia-Ming Lee, Shao-Kai Liu, Ming-Ching Chang, Xin Li, Yu-Lun Liu, Chih-Chung Hsu·· 6 小时前
C$^4$：扩散语言模型中自适应采样与提前退出的协同框架
Commit Locally, Exit Globally: Coordinating Adaptive Sampling and Early Exit in Diffusion Language Models
AI 导读
研究者提出无需训练的框架 C$^4$，将局部解码加速与自适应全局提前退出结合，在 LLaDA 上校准一次即可迁移至全部 12 个零样本任务及 Dream。该框架在保持任务性能接近全解码的同时，将解码步数减少 64-95%，实现 2.6x-18.6x 的解码循环加速。
正文
Abstract:Diffusion language models predict tokens in parallel but still require many iterative forward passes. Adaptive sampling accelerates local token commitment, while recent global stopping methods seek to end generation early. Our trajectory analysis shows that short answers can stabilize early in decoding, whereas reasoning outputs often require more steps to settle. A fixed progress threshold therefore struggles to avoid both redundant computation on short-answer tasks and premature termination on reasoning tasks. We introduce C$^4$, a training-free framework that combines local decoding acceleration with adaptive global early exit. Confidence-Verified Early Exit (CVEE) determines when to end generation, skipping the remaining steps once its stopping criterion is met. It combines confidence and temporal agreement to adapt stopping to the observed revision history. Commit-Core-Then-Confirm (CCTC) accelerates token commitment while generation continues, reducing the decoding steps needed to complete each block. It extends confidence-threshold parallel decoding with a boundary-anchored core and one-step confirmation of deferred predictions, constraining additional commitments to mitigate premature freezing under unresolved context. With CVEE calibrated once on LLaDA, the same numerical settings transfer across all 12 zero-shot tasks and to Dream without recalibration. Under this configuration, C$^4$ reduces decoding steps by 64-95% and achieves measured decoding-loop speedups of 2.6x-18.6x, with task performance close to full decoding. Component analyses show complementary contributions: early exit supplies most savings on short-answer tasks, while local acceleration contributes more on reasoning and code generation.
| Comments: | Code: this https URL. Project page: this https URL |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.28166 [cs.CL] |
| (or arXiv:2607.28166v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.28166 arXiv-issued DOI via DataCite |
Submission history
From: Chia-Ming Lee [view email]
[v1]
Thu, 30 Jul 2026 13:04:47 UTC (1,596 KB)
[v2]
Thu, 13 Aug 2026 09:19:39 UTC (1,540 KB)
[v3]
Wed, 14 Oct 2026 04:05:43 UTC (575 KB)
来源：arXiv：cs.CL · arxiv.org