arXiv：cs.LG· Yeongmin Kim, Arnaud Doucet, Andrew Campbell, Valentin De Bortoli, Thomas Mensink, David Ruhe·· 2 天前AI 评分38
Gumbel Straight Flow：将自回归模型蒸馏为一步流映射
Gumbel Straight Flow: Distilling Autoregressive Models into One-step Flow Maps
AI 导读
GSF 是一种连续流映射语言模型，利用预训练自回归语言模型的噪声-数据耦合，理论上证明 Gumbel 噪声与 one-hot token 序列之间的耦合会产生不相交的线性路径。该方法采用流映射半群目标，由 AR 教师模型直接引导切线（速度）条件，以增强高质量少步路径采样。在预训练和下游任务等多项基准上，GSF 可超越当前少步语言生成基线。
正文
Abstract:We present Gumbel Straight Flow (GSF), a continuous flow map language model that leverages the noise-data coupling of a pretrained autoregressive language (AR) model. We theoretically demonstrate that the coupling between Gumbel noise and one-hot token sequences induced by an autoregressive model yields non-intersecting linear paths connecting the noise to the sequence representations. To further enhance high-quality few-step path sampling, we use a flow map semigroup objective where the tangent (velocity) condition is guided directly by the AR teacher. Across various benchmarks, including pretraining and downstream tasks, GSF can outperform current few-step language generation baselines.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.00497 [cs.LG] |
| (or arXiv:2610.00497v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.00497 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: David Ruhe [view email]
[v1]
Wed, 30 Sep 2026 18:01:06 UTC (736 KB)
来源：arXiv：cs.LG · arxiv.org