LLM 后训练中的提示词干扰：为何 pass@k 优化可能降低 pass@1

View PDF HTML (experimental)

Abstract:Pass@$k$ is a widely used performance metric for verifiable large language model tasks, including mathematical reasoning, code generation, and short-answer reasoning. It defines success if any of $k$ independently sampled solutions passes a verifier. This multi-sample inference metric has motivated inference-aware fine-tuning methods that directly optimize pass@$k$. However, prior work reports a trade-off: pass@$k$ improves while pass@1 degrades under such methods. This trade-off is practically important because pass@1 often remains a hard operational constraint due to latency and cost budgets, imperfect verifier coverage, and the need for a reliable single-shot fallback. We study the origin of this trade-off and provide a theoretical characterization of when pass@$k$ policy optimization can reduce pass@1 through gradient conflict induced by prompt interference. This interference arises because shared model parameters couple prompts, so improving performance on some prompts can adversely affect others. We show that pass@$k$ policy gradients can conflict with pass@1 gradients because pass@$k$ optimization implicitly reweights prompts toward low-success prompts; when these prompts are negatively interfering, their upweighting can rotate the pass@$k$ update direction away from the pass@1 direction. We illustrate our findings with large language model experiments on verifiable mathematical reasoning and coding tasks.

Comments:	NeurIPS 2026
Subjects:	Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Cite as:	arXiv:2602.21189 [cs.LG]
	(or arXiv:2602.21189v3 [cs.LG] for this version)
	https://doi.org/10.48550/arXiv.2602.21189 arXiv-issued DOI via DataCite

Submission history

From: Anas Barakat [view email]
[v1] Tue, 24 Feb 2026 18:43:08 UTC (715 KB)
[v2] Thu, 26 Feb 2026 18:11:36 UTC (717 KB)
[v3] Tue, 13 Oct 2026 04:24:29 UTC (2,251 KB)