arXiv：cs.CL· Justin Jung·· 3 小时前AI 评分37
噪声化提示词：连续扩散语言模型中的条件 token 加噪训练
Noise Your Prompt: Noising Conditioning Tokens in Continuous Diffusion Language Models
AI 导读
研究者对连续扩散语言模型的标准做法做出单行修改：训练时也对条件提示词 token 加噪，从而在数独和 N-Queens 等组合推理任务上获得更好泛化，Sudoku Hard 求解率从 3.73% 提升至 24.65%，10x10 N-Queens 覆盖率达 73.79%。该方法在 Gigaword 摘要等小数据集自然语言生成上有改善，但未迁移到开放式对话生成；默认不增加推理成本，代码已公开。
正文
Abstract:We revisit a standard accepted practice in the continuous diffusion language model
literature of fixing conditioning prompt tokens clean during training.
We make a very simple modification: also noise the conditioning prompt tokens during training.
We demonstrate that under this modified training objective, we achieve better generalization
in combinatorial reasoning tasks such as Sudoku and N-Queens, with the largest gains on harder variants
($3.73\% \to 24.65\%$ solve rate on Sudoku Hard), and increased diversity of generated solutions ($50.60\% \to 73.79\%$ coverage on
10x10 N-Queens). We also show measurable improvements to natural language generation quality
in modest dataset regimes with Gigaword summarization, but notably demonstrate that gains do not
transfer to all natural language tasks (e.g open ended dialogue generation).
Our method is a single line change to the training objective, requires no additional inference costs by default,
and provides the flexibility of classifier-free guidance inspired guided sampling. Our
\href{this https URL} {code} is publicly available.
| Comments: | Published in Transactions on Machine Learning Research (TMLR), 2026 |
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.09145 [cs.LG] |
| (or arXiv:2610.09145v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.09145 arXiv-issued DOI via DataCite (pending registration) |
|
| Journal reference: | Transactions on Machine Learning Research (2026) |
Submission history
From: Justin Jung [view email]
[v1]
Tue, 6 Oct 2026 21:38:52 UTC (1,253 KB)
来源：arXiv：cs.CL · arxiv.org