arXiv：cs.AI· Shijing Hu, Xuancheng Ren, Zhihui Lu, Pan Zhou·· 3 小时前
树形推测解码新方法 TEV 与 ExitTrain：验证延迟降 15%，端到端提速 14%
Where Draft Trees Lose Target Mass: Exit-Guided Speculative Decoding
AI 导读
研究提出 Tree Exit Verification（TEV）与 ExitTrain：前者是基于退出节点决策和奖励 token 决策的精确层级并行验证流程，后者利用退出定律识别缺失的目标概率，为推理时草稿树提供节点级反馈。在对话、代码和数学推理任务中，ExitTrain 将平均输出块长度提升 13%，TEV 将验证阶段延迟降低 15%，相比 DDTree 实现 14% 的端到端加速。
正文
Abstract:Tree-based speculative decoding verifies multiple draft continuations in one target-model pass, but finite trees built from draft scores face a fundamental draft-target mismatch. We ask whether better exact verification can increase acceptance on a fixed tree and how target feedback can improve the tree itself. Through a target-flow view, we identify a canonical exit law and prove that one plus target coverage sharply bounds the expected output-block length, including the bonus token, of any exact path verifier. All optimal verifiers share the same exit and bonus-token law, already attained by representative predraw-and-follow and sequential residual verifiers. This yields Tree Exit Verification (TEV), an exact, level-parallel procedure using one exit-node decision and one bonus-token decision. The exit law also identifies missing target probability, providing node-level feedback for Exit-Guided Draft-Tree Training (ExitTrain) on inference-time draft trees. Experiments across dialogue, code, and mathematical reasoning validate fixed-tree equivalence: ExitTrain increases average output-block length by 13%, while TEV reduces verifier-stage latency by 15%, yielding a 14% end-to-end speedup over DDTree. Our results distinguish two opportunities: better draft trees for higher acceptance and more direct verification for lower latency. Code: this https URL.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.11750 [cs.AI] |
| (or arXiv:2610.11750v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.11750 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shijing Hu [view email]
[v1]
Thu, 8 Oct 2026 11:40:13 UTC (297 KB)
来源：arXiv：cs.AI · arxiv.org