arXiv：cs.LG· Zihan Shen, Qi Liu, Zixuan Yang, Yiqun Chen, Chenglong Zhao, Xiaozhao Wang, Lei He·· 2 天前AI 评分34
MatrixReward：用评分矩阵为开放式生成构建奖励
MatrixReward: Reward from Rubric Matrix for Open-Ended Generation
AI 导读
MatrixReward 通过在每个评分标准下两两比较采样回答，构建 rollout-by-rubric 胜率矩阵来生成奖励，并利用各列离散度与列间相关性得到数据相关的评分标准权重。在 Qwen3-8B 上、四个开放式问答基准测试中，MatrixReward 平均得分 63.02，比最强基线高出约 2.0%。
正文
Abstract:Open-ended query generation lacks standard answers, thus necessitating an effective reward mechanism. Pointwise scoring rubrics provide limited information about the relative quality of sample answers under the same prompt; merging multiple rubric judgments into a single score may also mask the differences between these answers. We propose MatrixReward, which constructs rewards from a rollout-by-rubric win-rate matrix obtained by comparing every pair of sampled responses under each rubric. The spread of each matrix column captures how strongly that rubric distinguishes the current rollouts, while correlations between columns reveal rubric repetition; together, these statistics yield data-dependent rubric weights. We combine these weights with the prior weights of rubrics. After column normalization and weighting, the observed per-rubric maxima and minima define positive and negative ideal profiles. Each rollout's distances to these two ideals determine its relative-closeness quality reward. Evaluated using Qwen3-8B on four open-ended query-answering benchmarks, MatrixReward achieves an average score of 63.02, outperforming the strongest baseline by approximately 2.0%. These results support the idea that matrices derived from relative comparisons can be used to construct rewards more reasonably for open-ended generative reinforcement learning.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML) |
| Cite as: | arXiv:2610.00389 [cs.LG] |
| (or arXiv:2610.00389v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.00389 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zihan Shen [view email]
[v1]
Wed, 30 Sep 2026 10:42:54 UTC (602 KB)
来源：arXiv：cs.LG · arxiv.org