arXiv：cs.AI· Roman Belaire, Arunesh Sinha, Pradeep Varakantham·· 10 小时前
用分层强化学习进行多轮对话红队测试
There are Levels to It: Red Teaming Dialogue With Hierarchical Reinforcement Learning
AI 导读
研究提出一种新的红队测试范式，将红队测试形式化为分层强化学习框架中的马尔可夫决策过程，通过 token 级 harm reward 训练生成式智能体学习多样化的多轮攻击，持续发现可绕过基线的漏洞。该方法在稀疏奖励与长时程规划挑战下取得新的 SOTA，将 LLM 红队测试重构为基于轨迹的原则性过程。
正文
Abstract:Red teaming is essential for securing Large Language Models, yet current automated methods remain limited by templates and single-turn attacks. To simulate the complex, interactive nature of real-world adversarial attacks, we introduce a novel red teaming paradigm designed to maximize expected cumulative harm through strategic interaction. By formalizing red teaming as a Markov Decision Process in a hierarchical reinforcement learning framework, we navigate the challenges of sparse rewards and long-horizon planning. Our generative agent learns diverse, multi-turn attacks using a token-level harm reward, consistently uncovering vulnerabilities that bypass baselines. This approach achieves a new state of the art and reframes LLM red teaming as a principled, trajectory-based process.
| Comments: | NeurIPS 2026 |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2508.04451 [cs.LG] |
| (or arXiv:2508.04451v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2508.04451 arXiv-issued DOI via DataCite |
Submission history
From: Roman Belaire [view email]
[v1]
Wed, 6 Aug 2025 13:52:00 UTC (2,043 KB)
[v2]
Tue, 13 Oct 2026 01:53:44 UTC (1,893 KB)
来源：arXiv：cs.AI · arxiv.org