arXiv：cs.CL· Lin Yao·· 4 小时前
CONDOR：用耦合噪声蒸馏让扩散语言模型一步生成连贯 token 块
A Ticket from Marginals to Joints: Coupled-Noise Distillation for One-Step Block Generation in Diffusion Language Models
AI 导读
研究者提出 CONDOR（Coupled-Noise Distillation for One-Step Readout），通过从零训练将不同噪声样本映射到不同的连贯 token 块，实现扩散语言模型一次前向传播生成完整 token 块。
正文
Abstract:Can a diffusion language model generate a coherent token block in one forward pass? Masked models already predict every position at once, but each prediction is the marginal distribution given the visible context, so the tokens can be mutually inconsistent and later steps revise those already committed. We introduce CONDOR (Coupled-Noise Distillation for One-Step Readout), trained from scratch to map different noise samples to different coherent blocks. Initially, random noise is not naturally paired with a target. Winner-take-all supervision lets different samples specialize, and self-distillation trains the one-pass output to match the refined coherent block. TinyStories experiments show diverse, coherent continuations over successive blocks, one forward pass each. Qualitative MNIST experiments show that the same approach can extend to multimodal generation, such as text-to-image and unconditional text-and-image generation.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2609.06324 [cs.CL] |
| (or arXiv:2609.06324v5 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2609.06324 arXiv-issued DOI via DataCite |
Submission history
From: Lin Yao [view email]
[v1]
Sun, 6 Sep 2026 01:09:14 UTC (767 KB)
[v2]
Fri, 11 Sep 2026 18:39:03 UTC (330 KB)
[v3]
Fri, 18 Sep 2026 09:46:15 UTC (346 KB)
[v4]
Wed, 30 Sep 2026 10:38:54 UTC (291 KB)
[v5]
Wed, 7 Oct 2026 19:40:55 UTC (292 KB)
来源：arXiv：cs.CL · arxiv.org