arXiv：cs.AI· Yiran Wu, Andrew Zhao, Yurui Chang, Qingyun Wu, Xuan Feng·· 11 小时前
超越二元可验证奖励：面向 SWE Agent 的行为自奖励方法 BSR
Beyond Binary Verifiable Rewards: Behavioral Self-Rewarding for SWE Agents
AI 导读
针对 RLVR 二元奖励无法区分调试质量的问题，研究者提出行为自奖励（BSR），复用策略本身对同一任务、同一验证结果下轨迹的可观测调试行为做两两比较，聚合成任务局部分数并转为有界辅助奖励，在不新增环境交互的前提下保留验证器的通过/失败排序。在 SWE-bench Verified 和 SWE-bench-Live 上，Qwen3-14B 与 Qwen3-32B 配置均取得一致提升。
正文
Abstract:Reinforcement learning with verifiable rewards (RLVR) provides reliable supervision for software engineering agents through executable tests, but binary rewards fail to distinguish trajectories with substantially different debugging quality. We investigate whether the current policy can judge its own trajectories and use these judgments to improve task performance without a separately trained reward model. To this end, we introduce Behavioral Self-Rewarding (BSR), a reward-refinement method that reuses the policy to compare observable debugging behavior among trajectories from the same task with the same verifier outcome. BSR aggregates pairwise preferences into task-local scores and converts them into bounded auxiliary rewards, introducing reward contrast while preserving the verifier-defined pass/fail ordering. This provides additional supervision from already-collected trajectories without requiring new environment interactions. Experiments on SWE-bench Verified and SWE-bench-Live show consistent improvements across the evaluated Qwen3-14B and Qwen3-32B configurations, supporting using the policy's own comparative judgments as a complementary learning signal for training software engineering agents.
| Subjects: | Artificial Intelligence (cs.AI); Software Engineering (cs.SE) |
| Cite as: | arXiv:2610.12598 [cs.AI] |
| (or arXiv:2610.12598v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.12598 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yiran Wu [view email]
[v1]
Thu, 8 Oct 2026 18:21:10 UTC (591 KB)
来源：arXiv：cs.AI · arxiv.org