arXiv：cs.AI· Yufeng Tian, Jiahao Tang, Minghao Wu·· 16 小时前
学习后执行：基于动作特定信用分配提升策略能力
Learning after Execution with Action-Specific Credit
AI 导读
研究提出"学习后执行"方法，将程序执行结果作为中间状态而非终端反馈，为同一策略创造额外的学习交互机会。通过动作特定信用分配，初始动作从最终轨迹结果获得信用，纠正动作则针对中间状态的改进获得信用。在 Image-to-TikZ 任务 945 个留出样本上，单次调用渲染成功率达 92.70%，高于持续单阶段 GRPO 的 88.25%。
正文
Abstract:Execution is often used as terminal feedback: it scores a generated program, but the resulting state is not used for another learning action. Learning after execution instead uses the execution result as an intermediate state that supports an additional learning interaction for the same policy. This creates a credit-assignment problem because the two actions share the same final objective but play different roles in the trajectory. Accordingly, action-specific credit assigns the initial action credit from the final trajectory outcome and the corrective action credit for improvement over the intermediate state. Image-to-TikZ serves as an executable visual testbed for this setting. On 945 held-out examples, learning after execution strengthens the shared policy under single-call inference and also produces a post-execution corrective capability. Single-call render success reaches 92.70\%, compared with 88.25\% for continued single-stage GRPO after the same number of additional training iterations. This corrective capability is not reproduced by continued single-stage training and depends on the intermediate state. Alternative credit definitions also induce different group-relative preferences over the same trajectories. Together, these results suggest that executable tasks contain useful learning structure beyond a terminal reward. Code is available at this https URL.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.14573 [cs.AI] |
| (or arXiv:2610.14573v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.14573 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yufeng Tian [view email]
[v1]
Sat, 10 Oct 2026 04:45:16 UTC (1,638 KB)
来源：arXiv：cs.AI · arxiv.org