arXiv：cs.AI· Wenfeng Wang, Jianrui Shi, Mingang Guo, Jiayuan Ma, Yuqi Lu, Jiajun Sun, Guojiang Zhao, Jing Li·· 16 小时前
Trace-to-Architecture：面向 harness 修订的能力瓶颈诊断框架
Trace-to-Architecture: Capability Bottleneck Diagnosis for Harness Revision
AI 导读
研究者提出 Trace-to-Architecture（T2A）框架，利用执行轨迹判断 LLM agent 失败时该改 harness 的哪个部分、改动何时生效，并通过新执行证据迭代诊断与修订。
正文
Abstract:Improving an LLM agent requires knowing what to change when it fails. Similar failures can require different fixes, and resolving one problem can expose another. We introduce Trace-to-Architecture (T2A), a framework that uses execution trajectories to determine what to change in an agent's harness and when those changes apply. A development agent diagnoses failures using execution evidence from successful and failed tasks, revises the corresponding harness components, and then checks whether target tasks improve, selected successful tasks remain successful, and the edits are actually triggered. New execution evidence guides subsequent diagnosis and revision. Across ALFWorld, ScienceWorld, and AppWorld, T2A achieves the highest aggregate score for all three evaluated task agents, exceeding their respective strongest baselines by 6.2--30.3 points. We also integrate T2A's diagnosis and validation procedures into Meta-Harness, improving the resulting harnesses' mean ScienceWorld score by 21.6 points from the same starting harness.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.16725 [cs.AI] |
| (or arXiv:2610.16725v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.16725 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jianrui Shi [view email]
[v1]
Mon, 12 Oct 2026 12:56:13 UTC (1,884 KB)
来源：arXiv：cs.AI · arxiv.org