arXiv：cs.CL· Huijuan Wang, Chufan Shi, Cheng Yang, Yaokang Wu, Taylor Berg-Kirkpatrick, Xuezhe Ma·· 4 小时前AI 评分42
统一多模态模型中的递归自我改进（RSI）
Recursive Self-Improvement in Unified Multimodal Models
AI 导读
研究者提出递归跨能力自我改进（RSI），让统一多模态模型的文本与视觉能力互相生成训练数据：模型生成图像并读取以发现不足，再编写程序，由执行结果对照规格验证。在图表任务上，四轮 RSI 将改写措辞请求的得分从 45.7% 提升至 60.2%，持续训练仅 46.3%；验证程序占比从 48.9% 升至 95.2%，读取器对编辑图像的准确率从 55.6% 升至 87.4%。
正文
Abstract:Unified multimodal models (UMMs) understand and generate both text and images, which lets a model produce its own training data. Existing self-improvement in UMMs keeps supervision on the visual side, where image understanding judges image generation. We propose recursive cross-capability self-improvement (RSI), a training loop in which the text and visual abilities of a UMM supply training data for one another. In each round, the model generates images and reads them to find where it falls short. It then writes programs aimed at these shortcomings, and execution verifies every result against its specification. Verified renders train image generation, while labeled renders and the model's own correct programs train visual understanding and program writing. Program execution thus acts as a source of truth outside the model, so errors do not accumulate across rounds. We study RSI on charts and build BasicChartBench to evaluate open models early in training. On requests worded differently from training, four rounds of RSI raise the score from 45.7% to 60.2%, while continued training stays at 46.3%. Verified construction carries most of the gain, and targeting the model's failures adds 3.5%. Along the way, the share of verified programs rises from 48.9% to 95.2%, and the reader's accuracy on edited renders rises from 55.6% to 87.4%.
| Subjects: | Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2610.03002 [cs.CL] |
| (or arXiv:2610.03002v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.03002 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Huijuan Wang [view email]
[v1]
Fri, 2 Oct 2026 08:34:21 UTC (518 KB)
来源：arXiv：cs.CL · arxiv.org