arXiv：cs.LG· Junze Zhu, Weihao Chen, Zhiyang Zhou, Zhen Wu, Xinyu Dai·· 4 小时前
Harness Policy Distillation：让 Agent 策略对齐运行时语义
Harness Policy Distillation: Aligning Agent Policies with Runtime Semantics
AI 导读
HPD 通过回放参考轨迹恢复运行时语义并表示为控制文档，让教师生成经 harness 验证的可执行动作，将运行时控制语义与动作联合训练进模型策略。在 TRAJECT-Bench 配合 Pi harness 上，Qwen3-8B 经 HPD 训练后自主任务成功率达 22.98%、运行时合规率 49.23%，而未适配模型仅为 3.09% 和 19.90%。
正文
Abstract:Deploying a general-purpose model as an agent means running it inside a product-specific runtime harness that decides which actions are executable, how state advances, and when the loop may terminate. Ordinary trajectory supervision shows which action was taken but leaves the runtime semantics behind that action implicit. We introduce Harness Policy Distillation (HPD), which makes these runtime control semantics an explicit training target alongside actions to align the harness policy. HPD replays reference trajectories through the harness to recover the runtime semantics that are implicit in ordinary trajectory data, and represents them as control documents at each model-call boundary. These documents provide privileged information for a teacher to generate actions, while harness verification ensures that the resulting targets are executable under the runtime contract. We jointly train a shared model on the runtime semantics recovered from control-state privileged information and native history, thereby internalizing these semantics into the model policy. At deployment, the trained model uses the original harness interface without requiring the control document or any additional supplement. On TRAJECT-Bench with the Pi harness, Qwen3-8B trained with HPD reaches 22.98% autonomous task success and 49.23% runtime compliance, against 3.09% and 19.90% for the unadapted model. Harness-verified teacher actions substantially improve the native deployment policy, while joint control-state supervision provides further gains in task completion and runtime compliance. HPD turns executable runtime semantics into dense supervision for learning harness-compatible actions at each decision boundary.
| Comments: | 23 pages, 9 figures |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.20261 [cs.LG] |
| (or arXiv:2610.20261v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20261 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Junze Zhu [view email]
[v1]
Wed, 14 Oct 2026 14:28:02 UTC (1,720 KB)
来源：arXiv：cs.LG · arxiv.org