Continual Graph Multi-Agent Reinforcement Learning：CGMARL 框架与 GRAFO 基准发布

View PDF HTML (experimental)

Abstract:In Continual Multi-Agent Reinforcement Learning (CMARL), agents learn cooperative policies across sequences of tasks, aiming to adapt effectively to new tasks while preserving the ability to solve previously encountered ones. In many applications, tasks differ in their underlying structure, which can represent, for example, distinct operational conditions or target configurations (e.g., different network topologies in power grids or arrangements in formation control). Existing CMARL methods lack dedicated mechanisms to leverage this structural information when learning new tasks, failing to promote transfer and mitigate forgetting. To fill this gap, we propose Continual Graph Multi-Agent Reinforcement Learning (CGMARL), a novel framework for CMARL problems in which task sequences are mapped into a series of attributed graphs, each modeling a task-specific structure. In CGMARL, each graph determines the environment dynamics (next states and/or rewards) and the number of agents for the corresponding task. Then, we present Graph-based Formation (GRAFO), the first CGMARL benchmark, and show how forgetting arises in this setting. Finally, to address this limitation, we propose Frozen Graph Encoder (FROG), a method that relies on a frozen graph backbone to preserve past structural information in graph-based CMARL policies. Experiments on GRAFO show that pairing FROG with existing CL methods substantially improves performance on multiple CGMARL scenarios.

Subjects:	Machine Learning (cs.LG)
Cite as:	arXiv:2610.10302 [cs.LG]
	(or arXiv:2610.10302v1 [cs.LG] for this version)
	https://doi.org/10.48550/arXiv.2610.10302 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Tommaso Marzi [view email]
[v1] Wed, 7 Oct 2026 15:56:31 UTC (436 KB)