arXiv：cs.LG· Yanjun Chen, Yirong Sun, Hanlin Wang, Jinghan Wang, Xinming Zhang, Xiaoyu Shen, Wenjie Li, Wei Zhang·· 4 小时前AI 评分44
轨迹即状态：LLM 智能体团队的精确信用分配
The Trace Is the State: Exact Credit Assignment for LLM Agent Teams
AI 导读
论文提出 C3（credit assignment by counterfactual continuation），在共享上下文的 LLM 智能体团队中通过替换决策点的一条消息并延续运行至终端奖励，实现无偏且精确到蒙特卡洛误差的信用分配。
正文
Abstract:Credit assignment for a team of LLM agents, what each message was worth, has mostly been treated as prediction: a learned critic, a trajectory-level score, or an agent ablation stands in for the counterfactual that defines credit, which is rarely run. Teams that communicate through a shared context are different: when everything a downstream agent reads is written into the trace, the trace is the state, and that counterfactual can be executed. A credit signal can then be judged as any estimator is: by bias, variance, and agreement with an independent reference. C3, credit assignment by counterfactual continuation, substitutes one message at a decision point and continues the run to the terminal reward, so its credit is unbiased, exact up to Monte Carlo error. Given the sampled alternatives, that error's variance follows a derived law with no term for the number of agents, and the observed noise follows the law on 6 workflows of 2 to 10 decision points. On the 2-agent chain, from 4 continuations, C3 ranks alternatives at 0.69 rank correlation against a 16-continuation reference, near the 0.73 at which 2 such references agree; a critic trained on the same rollouts reaches 0.29. Used as the advantage in training a 2-agent team, C3 beats MAGRPO, the stronger baseline in our comparison, and spends 37% fewer training tokens than MAPPO, since only the messages downstream of a decision point are regenerated. When the trace is the state, credit need not be predicted; it can be exact.
| Comments: | v3: substantially revised, new title. 30 pages, 3 figures |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2603.06859 [cs.LG] |
| (or arXiv:2603.06859v4 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2603.06859 arXiv-issued DOI via DataCite |
Submission history
From: Yanjun Chen [view email]
[v1]
Fri, 6 Mar 2026 20:25:11 UTC (607 KB)
[v2]
Fri, 8 May 2026 13:18:07 UTC (481 KB)
[v3]
Sat, 26 Sep 2026 16:22:44 UTC (298 KB)
[v4]
Wed, 7 Oct 2026 06:33:01 UTC (257 KB)
来源：arXiv：cs.LG · arxiv.org