arXiv：cs.CL· Geonmo Gu, Byeongho Heo, HeeJae Jun, Yoohoon Kang, Sangmin Lee, Sangdoo Yun, Dongyoon Han·· 6 小时前AI 评分42
DLoop：循环式投机解码
DLoop: Looped Speculative Decoding
AI 导读
DLoop 是一种循环式投机解码方法，在验证前自适应执行多轮起草阶段，让草稿模型在保持置信时持续起草并一次性验证所有累积 token。配合 loop-aware 训练，草稿模型能在额外起草阶段可靠使用未验证 token 的隐藏状态。在 EAGLE-3、DFlash、Domino、DSpark 及多 token 预测模块上，DLoop 将实际加速提升 5% 至 41%，且保持无损解码。
正文
Abstract:Speculative decoding accelerates autoregressive generation in large language models. In each drafting stage, a lightweight draft model proposes tokens that the target model subsequently verifies. With increasingly capable draft models, we find that the target model frequently accepts all tokens produced in a drafting stage. A verification nevertheless follows each drafting stage, resulting in unnecessary target-model forward passes even when drafting could have continued. Adaptive draft length methods decide during decoding how many draft tokens precede a verification, but they raise the speedup only for autoregressive draft models. For a parallel draft model, drafting further requires target-model hidden states for draft tokens that have not been verified. We propose DLoop, a looped form of speculative decoding that adaptively performs multiple drafting stages before verification. DLoop continues drafting while the draft model remains confident and verifies all accumulated draft tokens together. Loop-aware training keeps the draft model reliable in the additional drafting stages by exposing it to its own hidden states for unverified draft tokens. By spending additional draft-model forward passes, DLoop reduces the number of target-model forward passes required for verification. Across diverse speculative decoding methods including EAGLE-3, DFlash, Domino, DSpark, and multi-token prediction modules, DLoop improves the wall-clock speedup by 5 to 41 percent while preserving lossless decoding. Code will be available at this https URL.
| Comments: | 22 pages |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.07659 [cs.CL] |
| (or arXiv:2610.07659v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.07659 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Geonmo Gu [view email]
[v1]
Tue, 6 Oct 2026 02:55:25 UTC (280 KB)
来源：arXiv：cs.CL · arxiv.org