arXiv：cs.LG· Addison Kalanther, Sanika Bharvirkar, Shankar Sastry, Chinmay Maheshwari·· 4 小时前
NePPO：面向一般和博弈多智能体强化学习的近势能策略优化
NePPO: Near-Potential Policy Optimization for General-Sum Multi-Agent Reinforcement Learning
AI 导读
研究者提出 NePPO（Near-Potential Policy Optimization），通过学习与玩家无关的势函数，使以该势能为共同效用的合作博弈的纳什均衡逼近原一般和博弈的纳什均衡。该方法引入新的 MARL 目标并用零阶梯度下降最小化该目标，输出近似纳什均衡策略，在实验中表现优于 IPPO 和 MAPPO 等主流基线。该工作已被 IEEE CDC 2026 接收。
正文
Abstract:Multi-agent reinforcement learning (MARL) is increasingly used to design learning-enabled agents that interact in shared environments. However, training MARL algorithms in general-sum games remains challenging: learning dynamics can become unstable, and convergence guarantees typically hold only in restricted settings such as two-player zero-sum or fully cooperative games. Moreover, when agents have heterogeneous and potentially conflicting preferences, it is unclear what system-level objective should guide learning. In this paper, we propose a new MARL pipeline called Near-Potential Policy Optimization (NePPO) for computing approximate Nash equilibria in mixed cooperative--competitive environments. The core idea is to learn a player-independent potential function such that the Nash equilibrium of a cooperative game with this potential as the common utility approximates a Nash equilibrium of the original game. To this end, we introduce a novel MARL objective such that minimizing this objective yields the best possible potential function candidate and consequently an approximate Nash equilibrium of the original game. We develop an algorithmic pipeline that minimizes this objective using zeroth-order gradient descent and returns an approximate Nash equilibrium policy. We empirically show the superior performance of this approach compared to popular baselines such as IPPO and MAPPO.
| Comments: | Accepted to the IEEE Conference on Decision and Control (CDC) 2026 |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Science and Game Theory (cs.GT) |
| Cite as: | arXiv:2603.06977 [cs.LG] |
| (or arXiv:2603.06977v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2603.06977 arXiv-issued DOI via DataCite |
Submission history
From: Addison Kalanther [view email]
[v1]
Sat, 7 Mar 2026 01:35:52 UTC (204 KB)
[v2]
Sat, 4 Apr 2026 01:57:57 UTC (495 KB)
[v3]
Tue, 13 Oct 2026 18:05:59 UTC (495 KB)
来源：arXiv：cs.LG · arxiv.org