arXiv：cs.AI· Jiaming Su, Tengchao Yang, Ruikang Zhang, Zhengan Yan, Haoyu Sun, Linfeng Zhang·· 13 小时前
AnomalyAgent：用工具增强强化学习实现工业异常合成
AnomalyAgent: Agentic Industrial Anomaly Synthesis via Tool-Augmented Reinforcement Learning
AI 导读
AnomalyAgent 是一个具备自反思、知识检索与迭代优化能力的工业异常合成智能体，配备 Prompt Generation、Image Generation、Quality Evaluation、Knowledge Retrieval 和 Mask Generation 五个工具，并采用监督微调加强化学习的两阶段训练与三部分奖励机制。
正文
Abstract:Industrial anomaly generation is a crucial method for alleviating the data scarcity problem in anomaly detection tasks. Most existing anomaly synthesis methods rely on single-step generation mechanisms, lacking complex reasoning and iterative optimization capabilities, making it difficult to generate anomaly samples with high semantic realism. We propose AnomalyAgent, an anomaly synthesis agent with self-reflection, knowledge retrieval, and iterative refinement capabilities, aiming to generate realistic and diverse anomalies. Specifically, AnomalyAgent is equipped with five tools: Prompt Generation (PG), Image Generation (IG), Quality Evaluation (QE), Knowledge Retrieval (KR), and Mask Generation (MG), enabling closed-loop optimization. To improve decision-making and self-reflection, we construct structured trajectories from real anomaly images and design a two-stage training framework: supervised fine-tuning followed by reinforcement learning. This process is driven by a three-part reward mechanism: (1) task rewards to supervise the quality and location rationality of generated anomalies; (2) reflection rewards to train the model's ability to improve anomaly synthesis prompt; (3) behavioral rewards to ensure adherence to the trajectory. On the MVTec-AD dataset, AnomalyAgent achieves IS/IC-L of 2.10/0.33 for anomaly generation, 57.0% classification accuracy using ResNet34, and 99.3%/74.2% AP at the image/pixel level using a simple UNet, surpassing all zero-shot SOTA methods. The code is publicly available at this https URL.
| Comments: | 23 pages, 10 figures. Accepted at the ACM International Conference on Multimedia (ACM MM) 2026 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2604.07900 [cs.CV] |
| (or arXiv:2604.07900v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2604.07900 arXiv-issued DOI via DataCite |
Submission history
From: Jiaming Su [view email]
[v1]
Thu, 9 Apr 2026 07:17:00 UTC (7,165 KB)
[v2]
Fri, 9 Oct 2026 09:56:12 UTC (7,198 KB)
来源：arXiv：cs.AI · arxiv.org