arXiv：cs.CL· Juan Terven, Diana-Margarita C\'ordova-Esparza, Ruben Gomez, Carlos Miranda Rocha, Julio-Alejandro Romero-Gonz\'alez, Edgar Chavez-Urbiola·· 13 小时前
语言模型能学会何时拒答吗？Qwen3-14B 上的策略学习与校准答案选择
Can Language Models Learn When to Abstain? Policy Learning and Calibrated Answer Selection
AI 导读
研究在 Qwen3-14B 上对比了训练模型自身与训练独立答案选择器两种拒答方案，使用奖励正确答案、惩罚错误并允许弃答的评分。在 TriviaQA 上，学习式答案选择器将平均覆盖率从 20.2% 提升至 30.9%，已评分答案的汇总错误率从 23.0% 降至 14.5%（exact-match 评分）。
正文
Abstract:A useful language model should know when to answer and when to say "I don't know." Can we improve that decision by training the model itself or by training a separate component to check its answers? We study both approaches on Qwen3-14B for factual questions, using a score that rewards correct answers, penalizes mistakes, and allows abstention. Across three training seeds and two held-out confidence thresholds, a learned answer selector improves threshold-averaged utility over the original model's calibrated confidence filter. On TriviaQA, mean coverage increases from 20.2% to 30.9%, while the pooled error rate among scored answers decreases from 23.0% to 14.5% under exact-match grading. The tested reinforcement-learning changes do not establish consistent gains. On NQ-Open, the selector improves threshold-averaged utility over the filter but still performs worse than always abstaining. A partial human-reviewed audit preserves the TriviaQA finding. Our results support learning to select answers in this setting while showing that reliable abstention across datasets remains unresolved.
| Comments: | 17 pages, 2 figures, 10 tables |
| Subjects: | Computation and Language (cs.CL) |
| ACM classes: | I.2.7; I.2.6 |
| Cite as: | arXiv:2610.13821 [cs.CL] |
| (or arXiv:2610.13821v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.13821 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Juan Terven [view email]
[v1]
Fri, 9 Oct 2026 16:33:27 UTC (432 KB)
来源：arXiv：cs.CL · arxiv.org