arXiv：cs.AI· Wenxi Yang, Yuzhong Zhao, Baochang Zhang, Yunfan Liu, Qixiang Ye·· 16 小时前
Marvo：用 margin reward 实现视觉智能体自进化
Self-Evolving Visual Agent via Margin Reward
AI 导读
研究提出 Marvo，一种无需标注数据的自进化方法，让视觉智能体从无标注图像中学习工具调用。它由生成问答对的 challenger 和学习工具调用的 solver 两个智能体组成，并以"有无工具时 solver 表现之差"作为 margin reward 构建自适应课程。在智能体推理、视觉搜索、数学与通用现实推理等基准上，Marvo 平均准确率最高提升 7.1%。
正文
Abstract:Visual agents that interleave reasoning with tool invocation have posed a promising direction for multimodal tasks. However, learning effective tool invocation usually relies on curated question-answer pairs or expert demonstrations at scale, which are expensive to acquire. In this study, we propose Marvo, a simple-yet-effective self-evolution approach that enables learning tool invocation from unlabeled images. Marvo consists of two agents: (1) a challenger that generates question-answer pairs from unlabeled images, and (2) a solver that learns tool invocation by solving questions through reinforcement learning. To encourage effective tool invocation, a good training question should be difficult enough to warrant tool use, yet remain within the solver's capabilities. To this end, we introduce a margin reward to the challenger, defined as solver's performance difference with and without access to tools. With the margin reward, we create an adaptive curriculum that encourages the solver to learn effective tool invocation. Across benchmarks covering agentic reasoning, visual search, mathematical, and general real-world reasoning, Marvo improves average accuracy by up to 7.1\% --- demonstrating margin reward as a preferred way to scale question complexity without human supervision.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.15084 [cs.AI] |
| (or arXiv:2610.15084v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.15084 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Wenxi Yang [view email]
[v1]
Sat, 10 Oct 2026 17:33:56 UTC (3,668 KB)
来源：arXiv：cs.AI · arxiv.org