arXiv：cs.AI· Svar Rajankumar Patel, Kshitij Jerath·· 18 小时前
ADiCo：用极值搜索控制在线调节多智能体强化学习中的行为多样性
Optimizing Team Behavior via Extremum-seeking Control in Multi-agent Reinforcement Learning
AI 导读
研究者提出自适应多样性控制（ADiCo），一种双时间尺度框架，可在训练中在线调节多智能体强化学习的行为多样性：快时间尺度上 Diversity Control（DiCo）将系统多样性调控至目标值，慢时间尺度上用极值搜索控制器（ESC）仅凭平均回合回报调整该目标，无需显式模型或多样性-性能梯度的解析式。
正文
Abstract:In cooperative Multi-Agent Reinforcement Learning (MARL), agents must learn to coordinate their actions to achieve shared objectives. A key indicator of effective coordination is behavioral diversity, i.e., the agents' potential to enact distinct yet complementary control policies given the same set of observations. Although behavioral diversity can enable role specialization and richer joint behaviors, its appropriate level is highly context- and task-dependent and often costly to tune. We propose Adaptive Diversity Control (ADiCo), a two-timescale framework that adapts behavioral diversity online during training. On a fast timescale, Diversity Control (DiCo) regulates system diversity toward a target value. On a slower timescale, an Extremum Seeking Controller (ESC) adjusts this target using only mean episodic returns, without requiring an explicit model or analytical gradient of the diversity-performance landscape. We evaluate ADiCo on six configurations across two cooperative MARL tasks, where it maintains task-appropriate diversity levels: low diversity when homogeneous policies are beneficial and higher diversity when heterogeneous policies are required. Through this adaptation, ADiCo matches or exceeds the mean performance of DiCo (best grid-searched diversity target) and Unconstrained DiCo (diversity allowed to emerge without target regulation). These results demonstrate that incorporating classical feedback control into MARL training enables effective coordination by providing a systematic alternative to task-specific manual selection of the diversity target.
| Comments: | Accepted for publication at the 65th IEEE Conference on Decision and Control (CDC 2026), Hawaii, USA |
| Subjects: | Multiagent Systems (cs.MA); Artificial Intelligence (cs.AI); Robotics (cs.RO); Systems and Control (eess.SY) |
| Cite as: | arXiv:2610.18808 [cs.MA] |
| (or arXiv:2610.18808v1 [cs.MA] for this version) | |
| https://doi.org/10.48550/arXiv.2610.18808 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Kshitij Jerath [view email]
[v1]
Tue, 13 Oct 2026 17:46:27 UTC (981 KB)
来源：arXiv：cs.AI · arxiv.org