arXiv：cs.CL· Yangyi Shen, Pin-Jui Ku, Nicolai Oswald, Jean-Marie Lemercier, Yonggan Fu, Stefano Ermon, Jure Leskovec, Boris Ginsburg, Ante Juki\'c·· 16 小时前
One Block Ahead：提升扩散语言模型的采样器状态
One Block Ahead: Promoting Sampler State in Diffusion Language Models
AI 导读
研究提出免训练解码算法 One Block Ahead（OBA），在当前块去噪的同时并行去噪下一个块，关键在于将下一块的 token 与采样器状态（含自条件信号和去噪时间）一同提升。在 DiffusionGemma 上，OBA 使每次前向传播的 token 数平均提升 42.3%（从 15.64 增至 22.25），单请求去噪吞吐提升 16.3%，但准确率略有下降。
正文
Abstract:Block-diffusion language models denoise a whole block of token positions in parallel so each forward pass can emit many tokens. However, blocks are still denoised in strict sequence, a cost that within-block acceleration leaves untouched. Here, we investigate decoding blocks concurrently and propose One Block Ahead (OBA), a training-free decoding algorithm that denoises the next block alongside the current one. We demonstrate that for uniform-state diffusion LMs, where a block's denoising progress is not visible in its intermediate tokens, concurrent decoding is advantageous if the next block's tokens are promoted together with the corresponding sampler state, including the self-conditioning signal and the denoising time. We also derive a break-even criterion that relates the benefit of OBA to the relative cost of a paired-block forward pass. On DiffusionGemma, a state-of-the-art diffusion LM, OBA increases the tokens per forward pass by 42.3% (from 15.64 to 22.25) and the single-request denoising throughput by 16.3%, averaged over ten benchmarks, with a small accuracy regression.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.19510 [cs.CL] |
| (or arXiv:2610.19510v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19510 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yangyi Shen [view email]
[v1]
Wed, 14 Oct 2026 03:28:31 UTC (184 KB)
来源：arXiv：cs.CL · arxiv.org