arXiv：cs.AI· Junkai Chen, Yuhao He, Qianshan Wei, Junxiang You, Jingwen Shao, Junkai Lin, Zhongkai Yue, Xiaotian Ye, Zhengbo Jiao, Jiali Cheng, Zhijie Deng, Kening Zheng, Ruiqi Liu, Hadi Amiri, Yi Yu, Zhenan Sun, Qi Li, Ka-Ho Chow, Sijia Liu, Liang Wang, Jiaqi Li, Shu Wu·· 5 小时前
Open-MMUnlearning：统一 MLLM 遗忘方法与评估的开源框架
Open-MMUnlearning: Unifying Methods and Evaluation for MLLM Unlearning
AI 导读
研究者推出开源框架 Open-MMUnlearning，统一 MLLM 遗忘的目标模型准备、多模态数据处理、遗忘与评估流程，支持五大基准、四个模型家族的八款 MLLM 与十二种遗忘方法。
正文
Authors:Junkai Chen, Yuhao He, Qianshan Wei, Junxiang You, Jingwen Shao, Junkai Lin, Zhongkai Yue, Xiaotian Ye, Zhengbo Jiao, Jiali Cheng, Zhijie Deng, Kening Zheng, Ruiqi Liu, Hadi Amiri, Yi Yu, Zhenan Sun, Qi Li, Ka-Ho Chow, Sijia Liu, Liang Wang, Jiaqi Li, Shu Wu
Abstract:As multimodal large language models (MLLMs) become more capable and widely deployed, concerns about privacy and safety have become increasingly pressing. Machine unlearning offers one approach to addressing these concerns by removing designated information from trained models while preserving unrelated capabilities. However, fragmented implementations and evaluation protocols, incomplete robustness testing, and limited understanding of metric reliability make progress in MLLM unlearning difficult to assess systematically. We introduce Open-MMUnlearning, an open-source, extensible framework that integrates target-model preparation, multimodal data processing, unlearning, and evaluation through shared interfaces and structured configurations. The framework supports five benchmarks spanning privacy, safety, and copyright, eight MLLMs from four model families, and twelve unlearning methods. Its evaluation suite jointly assesses forgetting effectiveness, retained utility, and robustness to model interventions, adversarial inputs, and membership inference attacks. Using a common evaluation protocol, we compare ten representative unlearning methods. In this comparison, GD and MIP-Editor tie for the highest overall score: GD achieves the highest Forget Quality, while MIP-Editor preserves more Model Utility. We further introduce a metric meta-evaluation protocol that tests faithfulness using models with controlled exposure to target knowledge and robustness under quantization and relearning. Among the thirteen evaluated metrics, BLEU achieves the highest aggregate reliability score. KS-Test attains the highest faithfulness AUC but performs less well on robustness. Together, the framework and these findings support reproducible comparison of MLLM unlearning methods and systematic assessment of evaluation reliability.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.10358 [cs.AI] |
| (or arXiv:2610.10358v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.10358 arXiv-issued DOI via DataCite |
Submission history
From: Junkai Chen [view email]
[v1]
Wed, 7 Oct 2026 16:30:59 UTC (1,666 KB)
[v2]
Tue, 13 Oct 2026 10:15:51 UTC (1,669 KB)
来源：arXiv：cs.AI · arxiv.org