arXiv：cs.CL· Zhaoxiang Feng (UC San Diego), Lanxiang Hu (UC San Diego), Haoran Yuan (UIUC), Yu-Yang Qian (Nanjing University), Jihui Yang (Meta), Peng Zhao (Nanjing University), Yibo Zhu (StepFun), Tajana Rosing (UC San Diego), Hao Zhang (UC San Diego)·· 16 小时前
JetSpec++：前瞻并行草稿与自我修正的推测解码方法
JetSpec++: Foresight Parallel Drafting with Self Refinements
AI 导读
JetSpec++ 通过算法与系统协同设计，将推测解码的草稿接受率与生成开销同时优化，在 GB200 上单请求吞吐最高达 1,361 tokens/s，相比自回归解码提速最高 7.45 倍、相比 DSpark 提速 1.54 倍，七个基准上平均接受长度 8.94，并保持目标模型的输出分布不变。该方法结合块内自我修正、块间前瞻草稿与流水线式草稿验证三个组件，代码与模型已公开。
正文
Authors:Zhaoxiang Feng (1), Lanxiang Hu (1), Haoran Yuan (2), Yu-Yang Qian (3), Jihui Yang (4), Peng Zhao (3), Yibo Zhu (5), Tajana Rosing (1), Hao Zhang (1) ((1) UC San Diego, (2) UIUC, (3) Nanjing University, (4) Meta, (5) StepFun)
Abstract:Speculative decoding's effectiveness is governed by two quantities: the acceptance rate of drafted tokens and the cost of producing them. We show the two objectives can be co-optimized with algorithm-system co-design, and present JetSpec++, which retains a strong learned draft model while hiding its cost almost entirely. JetSpec++ combines three components: intra-block self refinement, which trains the draft model to iteratively correct its own parallel predictions within a block of tokens; inter-block foresight drafting, which begins the next block before the current one is verified; and pipelined drafting and verification, which overlaps the two stages so drafting's marginal wall-clock cost approaches zero. In comparison with state-of-the-art speculative decoding methods, JetSpec++ attains a wall-clock speedup of up to $7.45\times$ over autoregressive (AR) decoding and $1.54\times$ over DSpark, with up to 1,361 tokens per second for a single request on GB200, and an average acceptance length of 8.94 across seven benchmarks, while preserving the target model's output distribution. Our code and models are available at this https URL.
| Comments: | 31 pages, 4 figures, 16 tables. Zhaoxiang Feng and Lanxiang Hu contributed equally. Code and models: this https URL |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.19792 [cs.CL] |
| (or arXiv:2610.19792v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19792 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhaoxiang Feng [view email]
[v1]
Wed, 14 Oct 2026 08:37:27 UTC (324 KB)
来源：arXiv：cs.CL · arxiv.org