arXiv：cs.LG· Weixin Wang, Wei Deng, Anderson Schneider, Yuriy Nevmyvaka, Pan Xu, Andrew Bennett·· 4 小时前
无需训练提升 LLM 推理：解耦边缘锐化方法 DMS
Decoupled Marginal Sharpening for Training-Free Inference-Time Scaling
AI 导读
研究者提出 Decoupled Marginal Sharpening（DMS），一种无需更新模型权重或外部奖励模型的推理时扩展方法。DMS 先对推理路径锐化采样，再对聚合后的答案边缘概率做更强锐化，并配合 Fully Adapted Annealed SMC 缓解粒子路径坍缩。
正文
Abstract:We study how to improve LLM reasoning at inference time without updating model weights or using external reward models. Recent reward-free power-sampling methods define the sequence-level power target $\pi_\alpha(y\mid x)\propto p_0(y\mid x)^\alpha$, where $\alpha>1$, which increases the relative probability of completions that are already likely under the base model. Existing methods use Metropolis--Hastings or Sequential Monte Carlo (SMC) to approximately sample from this target. However, two challenges arise at different levels. At the target level, full-sequence sharpening can concentrate probability on a few exact reasoning--answer trajectories, suppressing alternative paths that support the same answer. At the sampler level, finite-particle SMC can suffer from path collapse when resampling repeatedly duplicates a few high-weight prefixes. To address these challenges, we propose Decoupled Marginal Sharpening (DMS), which separately controls how strongly the model favors particular reasoning paths and how strongly it favors final answers after aggregating support across those paths. At the target level, DMS first samples from a sharpened reasoning distribution, then marginalizes the model's answer probabilities over the resulting weighted reasoning population and sharpens the induced answer marginal. Separate exponents allow mild reasoning sharpening to encourage exploration of diverse paths, followed by stronger answer-marginal sharpening to concentrate probability on answers with greater aggregate support across those paths. At the sampler level, we develop Fully Adapted Annealed SMC, which combines an annealed sequence of power targets with locally fully adapted proposals and predictive ancestor resampling. Experiments on MATH500, GPQA-Diamond, HumanEval, and AIME24/25 across three base models show that DMS achieves the highest mean pass@1 among the compared methods in all tasks.
| Comments: | 17 pages, 3 figures, 6 tables |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.20483 [cs.LG] |
| (or arXiv:2610.20483v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20483 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Weixin Wang [view email]
[v1]
Wed, 14 Oct 2026 16:55:22 UTC (223 KB)
来源：arXiv：cs.LG · arxiv.org