arXiv：cs.AI· Guangcheng Zhu, Yingfan Ma, Xing Zheng, Haobo Wang, Shenzhi Yang, Kai Tang, Zhengqing Zang, Bowen Song, Weiqiang Wang, Gang Chen·· 22 小时前
CoCA：面向评分标准强化学习的标准感知信用分配
CoCA: Criterion-Aware Credit Assignment for Rubric-Based Reinforcement Learning
AI 导读
针对基于评分标准（rubric）的强化学习中标准级反馈被压缩为标量奖励、token 优势均匀分配的问题，研究者提出 CoCA，将评分标准视为信用分配的相关性线索，从组相对结果中导出标准级信用，并依据标准引发的分布偏移将其分配到各 token。在医学与科学基准上，CoCA 较标准 rubric-based RL 平均提升 +4.2%。
正文
Abstract:Rubric-based reinforcement learning (RL) evaluates responses against explicit criteria to construct rewards, offering a viable way to train large language models on open-ended tasks. However, standard methods compress criterion-level feedback into scalar rewards and assign uniform token advantages, leaving structured rubric supervision underutilized. Recent work seeks finer-grained guidance by treating rubrics as privileged information to derive self-distillation targets, but the resulting learning signal is not grounded in criterion outcomes and can lead to reward degradation. This motivates us to reframe the problem as criterion-aware credit assignment, where privileged rubric information serves as a relevance cue for credit allocation rather than a source of imitation targets. Accordingly, we propose CoCA, which derives criterion-level credits from group-relative outcomes and allocates them across tokens based on criterion-induced distribution shifts, yielding both outcome-grounded and fine-grained learning signals. Empirically, CoCA achieves an average gain of +4.2% over standard rubric-based RL across medicine and science benchmarks, demonstrating more effective learning on open-ended tasks.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.17733 [cs.LG] |
| (or arXiv:2610.17733v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.17733 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Guangcheng Zhu [view email]
[v1]
Tue, 13 Oct 2026 05:13:04 UTC (1,957 KB)
来源：arXiv：cs.AI · arxiv.org