arXiv：cs.AI· Yuxuan Zhang, Cong Wei, Penghui Du, Bo Li, Junwen Miao, Huaisong Zhang, Songcheng Cai, Yubo Wang, Dongfu Jiang, Yuyu Zhang, Changqian Yu, Ping Nie, Wenhu Chen, Kelsey R. Allen·· 17 小时前
RewardHarness：自进化智能体式后训练框架
RewardHarness: Self-Evolving Agentic Post-Training
AI 导读
RewardHarness 是一个免训练、自进化的智能体奖励框架，将奖励建模重构为上下文演化而非权重优化，仅用 100 条偏好演示即可迭代演化工具与技能库。它仅使用 EditReward 偏好数据 0.05% 的量，就在 EditReward-Bench K=2/3/4 和 GenAI-Bench 上达到 47.4% 平均准确率，超过 GPT-5 达 5.3 个百分点。
正文
Authors:Yuxuan Zhang, Cong Wei, Penghui Du, Bo Li, Junwen Miao, Huaisong Zhang, Songcheng Cai, Yubo Wang, Dongfu Jiang, Yuyu Zhang, Changqian Yu, Ping Nie, Wenhu Chen, Kelsey R. Allen
Abstract:Aligning generative models with human preferences remains a central challenge. While RLHF is widely adopted, it relies on large-scale preference data. This limitation is particularly evident in evaluating instruction-guided image edits, even though humans can often learn such preferences from only a few examples. We present RewardHarness, a training-free, self-evolving agentic reward framework that reframes reward modeling as context evolution rather than weight optimization. Instead of learning from large-scale annotations, RewardHarness aligns with human preferences by iteratively evolving a library of tools and skills from as few as 100 preference demonstrations. Given a source image, an edited image, and an editing instruction, an orchestrator agent selects the most relevant subset of tools and skills from the maintained library, and sub-agents invoke them to construct a reasoning chain that produces a reward score. By comparing predicted scores with ground-truth preferences and analyzing successes and failures in the reasoning process, the orchestrator automatically refines its library of tools and skills without additional human annotation. Using only 0.05% of the EditReward preference data, RewardHarness achieves 47.4% average accuracy on EditReward-Bench K=2/3/4 and GenAI-Bench, surpassing GPT-5 by 5.3 points. When used as a reward signal in FlowGRPO, RL-tuned models achieve 3.52 on ImgEdit-Bench. Code and model are available at the project page: this https URL
| Comments: | Project page: this https URL |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.08703 [cs.AI] |
| (or arXiv:2605.08703v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2605.08703 arXiv-issued DOI via DataCite |
Submission history
From: Yuxuan Zhang [view email]
[v1]
Sat, 9 May 2026 05:32:48 UTC (11,259 KB)
[v2]
Mon, 12 Oct 2026 10:20:42 UTC (23,202 KB)
来源：arXiv：cs.AI · arxiv.org