arXiv：cs.LG· Ioannis Anagnostides, Constantinos Daskalakis, Gabriele Farina, Noah Golowich, Tuomas Sandholm, Brian Hu Zhang·· 5 小时前AI 评分47
马尔可夫博弈中的标准型相关均衡：首个高效算法与 PPAD 完备性
Normal-Form Correlation in Markov Games
AI 导读
研究者给出有限时域马尔可夫博弈中计算标准型相关均衡（NFCE）的首个高效算法，在 n 个玩家、S 个状态、时域 H、每人至多 A 个动作下，可在 S(AH/ε)^O(n) 时间内算出 ε-NFCE，这是首个对 1/ε 和博弈描述均多项式的 NFCE 算法。
正文
Abstract:There has been a surge of recent work on correlated equilibrium concepts in Markov games. However, existing results focus on concepts weaker than normal-form correlated equilibria (NFCEs), leaving open the more challenging question of computing such equilibria, which goes back to the seminal work of Papadimitriou and Roughgarden (JACM'08). Here, we establish the first efficient algorithm for NFCEs in finite-horizon Markov games with a fixed number of players $n$. In particular, with $S$ states, horizon $H$, and at most $A$ actions per player, it computes an $\epsilon$-NFCE in time $S(AH/\epsilon)^{O(n)}$. This is the first algorithm polynomial in $1/\epsilon$ and the description of the game for NFCEs in an interesting class of problems beyond the normal-form setting. Moreover, under the usual assumption that recommendations are independent across states, we show PPAD-completeness---that is, computational equivalence to Nash equilibria---either in many-player games or when the precision is exponentially small.
The key idea behind our approach is to run backward induction on a sequence of auxiliary stage games, but with the twist that in each step we compute a constant-expectation correlated equilibrium. This is a natural refinement of correlated equilibrium in which the conditional expected payoff from obeying is independent of the recommendation. In fact, our reduction goes both ways, establishing an equivalence between constant-expectation CEs and NFCEs in Markov games. For a fixed number of players, we observe that a constant-expectation CE can be computed approximately by combining linear programming with suitable discretization. In contrast, it is PPAD-hard in i) polymatrix (many-player) games at constant precision, and ii) two-player games at exponentially small precision. The latter result follows from an unexpected connection to rank-2 two-player games.
| Subjects: | Computer Science and Game Theory (cs.GT); Computational Complexity (cs.CC); Data Structures and Algorithms (cs.DS); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.03621 [cs.GT] |
| (or arXiv:2610.03621v1 [cs.GT] for this version) | |
| https://doi.org/10.48550/arXiv.2610.03621 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ioannis Anagnostides [view email]
[v1]
Fri, 2 Oct 2026 17:18:51 UTC (31 KB)
来源：arXiv：cs.LG · arxiv.org