arXiv：cs.CL· Yi Ding, Mengyu Wang, Min Zhou, Menglin Yang·· 16 小时前
REWIND：用熵动态引导的测试时推理轨迹修复
REWIND: Uncertainty-Guided Trajectory Repair for Test-Time Reasoning
AI 导读
REWIND 是一个无需训练的控制器，利用 token 熵动态定位推理错误位置并定向重生成后续内容。在 MATH500 上，Qwen2.5-Math-7B 的错误回答平均 token 熵比正确回答高约 112%。该方法在三个模型系列、五个基准上超过所有基线，比最强基线高出 0.51-10.00 个百分点。
正文
Abstract:An early mistake in the generated reasoning can constrain all subsequent reasoning. Repeated-generation methods, like MCMC-based approaches, attempt to improve output quality by sampling multiple generations, but their effectiveness can be limited by randomly selecting where to regenerate. On MATH500 with Qwen2.5-Math-7B, we find that incorrect responses have substantially higher mean token entropy than correct responses, with an average within-question relative increase of approximately 112%, suggesting that entropy can signal reasoning errors. Motivated by this finding, we introduce REWIND, a training-free controller that uses entropy dynamics to propose targeted suffix regenerations. Across three model families and five benchmarks, REWIND outperforms all baselines, with gains of 0.51-10.00 percentage points over the strongest baseline. These results demonstrate the effectiveness of targeted, uncertainty-guided trajectory repair across models and tasks. Our code is available at this https URL.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.19561 [cs.CL] |
| (or arXiv:2610.19561v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19561 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yi Ding [view email]
[v1]
Wed, 14 Oct 2026 04:20:56 UTC (139 KB)
来源：arXiv：cs.CL · arxiv.org