arXiv：cs.AI· Sungjun Lim, Adrian Robert Minut, Nico Carlsen Daheim, Mohammad Emtiyaz Khan, Kyungwoo Song, Thomas M\"ollenhoff·· 19 小时前
Fenchel-Bregman 后验学习：用单次采样更新提升多样本推理覆盖率
Train for Many, Update with One: Improving Multi-Sample Reasoning via Fenchel-Bregman Posterior Learning
AI 导读
研究者提出有限 K 后验学习，让模型参数分布直接优化 K 个候选中至少一个正确的概率，而非最小化单次尝试的平均损失。其提出的 Fenchel-Bregman 方法每次更新只需一次后验采样，结合 IVON 实现 FBI，在多种递归与自回归推理任务上提升了候选覆盖率，达到相近 Pass@10 所需训练时间仅为直接蒙特卡洛基线的 12.3%。
正文
Abstract:Multi-sample reasoning measures candidate coverage, the probability that at least one of $K$ candidates is correct. However, minimizing the average loss of individual attempts does not directly optimize candidate coverage. We therefore formulate finite-$K$ posterior learning, which trains a distribution over solver parameters to reduce the probability that all $K$ independently generated candidates fail. To optimize this objective efficiently, we propose a Fenchel-Bregman (FB) approach that requires only one posterior draw per update, rather than the $K$ used by direct Monte Carlo optimization. FB first uses an exact Fenchel reformulation to separate problem-specific weights from the posterior update, enabling one-draw posterior updates. It then applies a closed-form Bregman update to track these weights through an exponential running average of observed failures, reusing the same draw. We instantiate FB with IVON, a variational optimizer for Gaussian posteriors over solver parameters, yielding Fenchel-Bregman IVON (FBI). Across diverse recursive and autoregressive reasoning tasks, FBI improves candidate coverage. In a matched comparison, it achieves comparable Pass@10 using 12.3% of the direct Monte Carlo baseline's measured training time per update.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.17735 [cs.LG] |
| (or arXiv:2610.17735v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.17735 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Sungjun Lim [view email]
[v1]
Tue, 13 Oct 2026 05:13:26 UTC (522 KB)
来源：arXiv：cs.AI · arxiv.org