arXiv：cs.LG· Dongsu Lee, Haoran Xu, Amy Zhang·· 3 小时前AI 评分32
SCOUT：通过分解价值梯度流实现测试时多智能体协调
Test-time Multi-agent Coordination by Decomposed Value Gradient Flow
AI 导读
研究者提出 SCOUT，首个将生成式基础模型与学习价值函数通过测试时动作精炼结合的离线多智能体强化学习框架。SCOUT 训练流匹配行为先验与分解价值函数两个解耦组件，测试时用 Stein 变分梯度下降将行为样本输运至高价值区域，输运步数实现自适应测试时扩展。在离散与连续离线 MARL 基准上，SCOUT 取得最佳平均性能，并在所有离线到在线配置中均有提升。
正文
Abstract:Offline multi-agent reinforcement learning (MARL) faces a persistent trade-off. Expressive generative policies can represent multi-modal coordination in the data, but cannot distinguish high-value regions, while value-optimized policies exploit the learned Q-function but collapse the multi-modal into a single dominant mode. A single agent's mode collapse can break joint coordination, and simultaneous drift across agents can push the joint policy into unseen regions of the action space. We propose scalable coordination via optimal unified transport (SCOUT), the first offline MARL framework to combine a generative foundation model with a learned value function through test-time action refinement. SCOUT trains two decoupled components: a flow-matching behavioral prior and a decomposed value function. At test-time, it transports behavioral samples toward high-value regions via Stein variational gradient descent. The number of transport steps controls adaptive test-time scaling, replacing a fixed regularization coefficient. Under the individual-global-max (IGM) principle, we prove a single-term KL bound on the joint soft-value gap that vanishes as transport converges, with an irreducible additive residual proportional to the IGM violation. Empirically, SCOUT achieves the best average performance across discrete and continuous offline MARL benchmarks and yields performance improvements in all offline-to-online configurations.
| Comments: | NeurIPS 2026 |
| Subjects: | Machine Learning (cs.LG); Robotics (cs.RO) |
| Cite as: | arXiv:2610.02554 [cs.LG] |
| (or arXiv:2610.02554v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.02554 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Dongsu Lee [view email]
[v1]
Thu, 1 Oct 2026 22:38:27 UTC (5,343 KB)
来源：arXiv：cs.LG · arxiv.org