arXiv：cs.LG· Zeyu Michael Li, William Xingxu Chen, Bingshuo Qian, Jiayin Liu, Xiang Cheng·· 4 小时前AI 评分36
ELF-REG：将连续扩散语言模型扩展至推理任务
ELF-REG: Scaling Continuous Diffusion Language Models to Reasoning Tasks
AI 导读
ELF-REG 将 Embedded Language Flows（ELF）扩展到数学推理与代码生成，通过冻结的 AR 教师模型监督中间去噪特征并提供联合去噪的全局表示。
正文
Abstract:Fully continuous diffusion language models (dLMs) denoise continuous representations without intermediate discretization, then decode all response tokens in parallel at the final step. Their performance on challenging reasoning tasks remains less established than that of autoregressive (AR) LLMs and masked dLMs. We scale Embedded Language Flows (ELF) to mathematical reasoning and code generation on GSM8K, MATH-500, HumanEval, and MBPP. We introduce ELF-REG, which improves learning with representation alignment and entanglement (REPA+REG), where a frozen AR teacher supervises intermediate denoiser features and supplies a global representation that is jointly denoised with the response. ELF-REG-L achieves 55.96% pass@1 on GSM8K at 64 network function evaluations (NFE), and 13.39% on MATH-500 and 22.56% on HumanEval at 128 NFE. It outperforms the evaluated comparable-scale dLMs in pass@1 on GSM8K and code, and improves MATH-500 pass@1 from 10.55% for the ELF-L baseline to 13.39% with ELF-REG-L. Without few-step training, the same task-specific checkpoints support strong low-NFE performance through early-stop, which decodes an intermediate clean prediction without completing the denoising trajectory. At 16 NFE, ELF-REG-L reaches 41.21% HumanEval pass@10, outperforming recent continuous dLMs of comparable scale.
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2609.29102 [cs.CL] |
| (or arXiv:2609.29102v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2609.29102 arXiv-issued DOI via DataCite |
Submission history
From: Zeyu Michael Li [view email]
[v1]
Thu, 24 Sep 2026 06:33:19 UTC (335 KB)
[v2]
Tue, 6 Oct 2026 17:43:50 UTC (381 KB)
来源：arXiv：cs.LG · arxiv.org