arXiv：cs.AI· Hanbin Hong, Shuang Wu, Shuya Feng, Nima Naderloui, Shenao Yan, Jingyu Zhang, Ali Arastehfard, Heqing Huang, Yuan Hong·· 10 小时前
SoK：系统化 LLM 提示词安全——越狱攻击与防御的分类体系、数据集与统一评测
SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defenses
AI 导读
研究系统化梳理 272 项 LLM 提示词安全研究，提出三套多标签分类体系并引入基于 ⟨M,A,D,S,J⟩ 配置元组的访问兼容评测协议，落地为开源平台 PromptSecurity。
正文
Abstract:Prompt-security claims for large language models (LLMs) are often compared as properties of individual attacks or defenses, although measured harmfulness depends on the target model, access setting, attack and defense configuration, prompt sample, and judger. We systematize 272 studies through three linked, multi-label taxonomies of jailbreak attacks, defenses, and model vulnerabilities. To make these dependencies explicit, we introduce an access-compatible evaluation protocol over configured $\langle M,A,D,S,J\rangle$ tuples and instantiate it in PromptSecurity, an open-source platform that records method settings, outputs, judgments, and query costs. We release two complementary datasets: JailbreakDB, with 445,752 jailbreak system-user prompt pairs and 1,094,122 benign prompts, and PromptSecurity-Eval, with 166,200 harmful-query and 9,200 benign-utility output records. We evaluate 11 target models, 20 attacks, nine defenses, and three automated judgers under a matched, single-turn protocol using 100 harmful queries and separate benign-utility tasks. Lower attack success can reflect failure to execute transformed requests rather than stronger safety; attack-category advantages can reverse when one method is excluded, and attack success varies with assistant choice. Defenses can reduce harmful outputs while degrading benign-task utility or causing backfire. Rejudging identical outputs also shows that measured attack success is sensitive to the behavior supplied to the judger, highlighting the need to report judger configurations alongside ASR. These findings motivate configuration-aware reporting rather than context-free method rankings.
Leaderboard: this https URL. Dataset: this https URL. GitHub: this https URL.
| Subjects: | Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2510.15476 [cs.CR] |
| (or arXiv:2510.15476v4 [cs.CR] for this version) | |
| https://doi.org/10.48550/arXiv.2510.15476 arXiv-issued DOI via DataCite |
Submission history
From: Hanbin Hong [view email]
[v1]
Fri, 17 Oct 2025 09:38:54 UTC (416 KB)
[v2]
Tue, 21 Oct 2025 07:40:00 UTC (416 KB)
[v3]
Sat, 4 Jul 2026 04:20:15 UTC (273 KB)
[v4]
Sat, 10 Oct 2026 03:22:07 UTC (272 KB)
来源：arXiv：cs.AI · arxiv.org