arXiv：cs.AI· Chenlu Ding, Jiancan Wu, Yanchen Luo, Tianyu Zhang, Yancheng Yuan, Xiang Wang, Xiangnan He·· 16 小时前
CUSE：面向多模态大模型的细粒度遗忘框架，结合反事实引导与同策略自蒸馏
CUSE: Fine-Grained Multimodal Unlearning with Counterfactual Guidance and On-Policy Self-Distillation
AI 导读
研究者提出多模态大模型遗忘框架 CUSE，通过移除目标相关证据构建反事实输入，利用 token 概率变化定位需要更强遗忘的 token，并以冻结的原模型为教师、在当前策略采样响应上蒸馏 token 级遗忘与保留目标。在 MLLMU-Bench 上，CUSE 将遗忘集 ROUGE-L 降低 33.6%，保留集 ROUGE-L 比最强基线高 1.2%。
正文
Abstract:Multimodal large language models (MLLMs) can reproduce sensitive, copyrighted, or unsafe information from their training data. Effective unlearning must remove target information while preserving non-target content, even within the same response. Existing methods typically apply uniform forgetting objectives to complete responses and rely on fixed, off-policy supervision, resulting in coarse token control and misalignment with the model's current behavior. We propose CUSE, a Counterfactual-guided Unlearning framework via on-policy Self-distillation for multimodal large language models. It first constructs counterfactual inputs by removing target-related evidence and uses the resulting changes in token probabilities to identify which tokens require stronger forgetting. It then distills token-level forgetting and retention targets on responses sampled from the current policy, with a frozen copy of the original model serving as the teacher model augmented with the corresponding forgetting- and retention-oriented privileged contexts. This design jointly determines what to forget and what to preserve. On MLLMU-Bench, CUSE reduces forget-set ROUGE-L by 33.6% while achieving 1.2% higher retain-set ROUGE-L than the strongest baseline.
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.14763 [cs.AI] |
| (or arXiv:2610.14763v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.14763 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Chenlu Ding [view email]
[v1]
Sat, 10 Oct 2026 09:37:30 UTC (19,741 KB)
来源：arXiv：cs.AI · arxiv.org