HarnessPro：用帕累托优化实现性能与成本权衡的递归 Harness 自我改进

View PDF HTML (experimental)

Abstract:Harness development plays a crucial role in building capable AI agents. Recent approaches improve harnesses through recursive self-improvement (RSI), but their performance gains often come with higher execution costs. We introduce HarnessPro, a label-free method for evolving harnesses to improve task performance while reducing execution cost. In this setting, candidate harnesses are evaluated using LLM-generated tests without access to ground-truth solutions or official tests. Specialized agents diagnose execution trajectories, plan updates with different performance-cost priorities, and implement them as candidate harnesses. Rather than returning a single harness, HarnessPro uses normalized weighted squared-distance selection to provide cost-focused (lite), balanced (medium), and performance-focused (heavy) harnesses for different deployment budgets. Selected harnesses enter the next iteration, where execution feedback guides further updates. Our experiments cover Terminal-Bench 2, SWE-Bench Pro, and ProgramBench, using GPT-5.5 with Codex and GLM-5.2 with Claude Code. Across these benchmarks using GPT-5.5 with Codex, the heavy mode of HarnessPro achieves an average pass-rate gain of 2.0 percentage points and an average per-task cost reduction of 26.1% relative to RHO. This advantage also holds when using GLM-5.2 with Claude Code. Code is available at this https URL

Comments:	41 pages, 4 figures, 4 tables, under review
Subjects:	Machine Learning (cs.LG); Computation and Language (cs.CL)
Cite as:	arXiv:2610.26770 [cs.LG]
	(or arXiv:2610.26770v1 [cs.LG] for this version)
	https://doi.org/10.48550/arXiv.2610.26770 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Zhuoshi Pan [view email]
[v1] Mon, 19 Oct 2026 17:27:38 UTC (587 KB)