arXiv：cs.CL· Jerrold Huang, Junwei Quan, Saiyue Lyu, Zihan Guo, Yingqiang Ge·· 16 小时前
SkillScreen：用决策模型高召回筛查恶意 Agent 技能
SkillScreen: High-Recall Screening of Malicious Agent Skills with Decision Models
AI 导读
SkillScreen 是一个紧凑决策模型，单次前向读取 SKILL.md 即可直接为 Agent 技能打分判定恶意性。在 MaliciousSkillBench 上，它面对未见攻击家族时以 99% 召回率仅误标 20-25% 良性技能，明显少于词级分类器和提示式 LLM 评审；相同训练协议下对未见来源的泛化也优于全量微调编码器，但陌生良性来源会显著推高误报。
正文
Abstract:Agent skills turn natural-language instructions into actions, creating a security challenge: legitimate automation and malicious behaviour can look deceptively similar. A practical screen must catch nearly every attack while keeping the review load manageable. We present SkillScreen, a compact decision model trained to score skill maliciousness directly from SKILL$.$md in a single forward pass. On MaliciousSkillBench, SkillScreen flags 20-25% of benign skills at 99% recall on unseen attack families, substantially fewer than the word-level classifier and prompted LLM judges tested. Under the same training protocol, it also generalizes better than a fully fine-tuned encoder to unseen sources. Yet unfamiliar benign sources sharply increase false positives, showing that recognizing new attacks does not ensure reliable screening of new skill collections. Our results demonstrate the value of task-trained decision models for high-recall skill screening. Project page: this https URL
| Comments: | 21 pages, including references and appendices |
| Subjects: | Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.17727 [cs.CR] |
| (or arXiv:2610.17727v2 [cs.CR] for this version) | |
| https://doi.org/10.48550/arXiv.2610.17727 arXiv-issued DOI via DataCite |
Submission history
From: Jerrold Huang [view email]
[v1]
Tue, 13 Oct 2026 05:03:43 UTC (639 KB)
[v2]
Wed, 14 Oct 2026 04:56:03 UTC (639 KB)
来源：arXiv：cs.CL · arxiv.org