跨越网络鸿沟：RL 智能体的 Sim-to-Sim 与 Sim-to-Real 迁移

View PDF HTML (experimental)

Abstract:Cyber attack agents are typically trained and evaluated within a single simulator, making it unclear whether learned policies transfer beyond the environments in which they were developed. This limitation hinders both deployment and fair comparison, as cyber simulators differ substantially in their state representations, observation models, and action spaces. In this paper, we study policy transfer across cyber environments and argue that simulator-to-simulator and simulator-to-real transfer can be viewed as instances of the same underlying alignment problem. We propose a framework that separates state alignment from action translation, enabling a policy trained in one environment to operate in another without retraining. We evaluate transfer across four cyber platforms, CyberBattleSim, NetSecGame, CyberWheel, and NASim, including emulated deployments in NASim. Our experiments show that zero-shot transfer is feasible, fully preserving source-policy performance in closely aligned environments and achieving 45.2% win rates when transferring policies whose source performance is 60.5%. In emulated virtual machine environments, transferred policies exhibit a Jensen-Shannon divergence of 0.085 from native policies, indicating strong behavioral similarity. Code and benchmarks are available at: this https URL.

Comments:	13 pages, 3 figures, 1st Workshop on Real-world AI Security and Engineering for Cybersecurity Systems (RAISE) 2026
Subjects:	Cryptography and Security (cs.CR); Machine Learning (cs.LG)
Cite as:	arXiv:2610.00759 [cs.CR]
	(or arXiv:2610.00759v1 [cs.CR] for this version)
	https://doi.org/10.48550/arXiv.2610.00759 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Sabrina Saika [view email]
[v1] Wed, 30 Sep 2026 21:54:29 UTC (2,734 KB)