arXiv：cs.AI· Haiwen Yi, Xinyuan Song·· 20 小时前
HarnessMDP：用离线强化学习让 LLM Agent 执行框架可学习
HarnessMDP: Learning to Control LLM Agent Harnesses with Offline Reinforcement Learning
AI 导读
HarnessMDP 将 LLM Agent 的执行框架（harness）控制建模为有限时域马尔可夫决策过程，让框架从不断变化的 Agent 状态中选择结构性执行动作，并基于收集的轨迹离线学习策略、上线后自适应控制新的执行过程。在编码任务、tau-bench、AgentBench 等评测中，HarnessMDP 的执行行为和最终任务表现均持续优于基线 harness。
正文
Abstract:The execution harness is a critical component of LLM agents, shaping how the agent interacts with its environment and organizes multi-step execution. Prior work has shown that improving tools, workflows, and other harness components can substantially improve agent performance. This suggests that the harness can be treated as a learnable control layer that adapts execution decisions from experience, rather than as a predetermined execution configuration. In this work, we introduce HarnessMDP, a learning-based framework that makes harness control itself learnable. HarnessMDP formulates agent execution as a finite-horizon Markov decision process, where the harness learns to select structural execution actions from the evolving agent state. The harness policy is learned offline from collected trajectories using task performance and deployed online to adaptively control new executions. Extensive experiments on coding tasks, tau-bench, AgentBench, and other agent evaluation settings show that HarnessMDP consistently improves execution behavior and final task performance over baseline harnesses. These results demonstrate the effectiveness of HarnessMDP and establish learnable harness control as a practical mechanism for improving LLM agents through better execution.
| Comments: | 13 pages, 4 figures |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.05458 [cs.LG] |
| (or arXiv:2607.05458v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2607.05458 arXiv-issued DOI via DataCite |
Submission history
From: Haiwen Yi [view email]
[v1]
Sun, 5 Jul 2026 22:11:18 UTC (1,093 KB)
[v2]
Fri, 9 Oct 2026 18:02:36 UTC (1,572 KB)
来源：arXiv：cs.AI · arxiv.org