arXiv：cs.LG· Jiashuo Zou, Xiaobo Xia·· 4 小时前
REFINE：在多模态噪声监督下学习该信任什么
Learning What to Trust in Multimodal Learning under Noisy Supervision
AI 导读
研究者提出 REFINE，一个多模态标签噪声检测框架，通过判别分析构建判别特征向量，为每个类别挑选噪声检测能力更强的可信表示空间，并结合融合与单模态表示选择可信样本。该框架用更干净的监督信号更新多模态分类器，减少误标注样本影响，在多种任务上优于基线方法，源代码将公开。
正文
Abstract:Multimodal classification processes and relates information from multiple modalities to achieve more accurate predictions. However, existing methods typically rely on high-quality ground-truth labels, which are difficult to obtain in real-world scenarios. While sample-selection methods for learning with noisy labels aim to identify correctly labeled examples from noisy data, traditional methods primarily focus on unimodal settings and fail to exploit multimodal information fully. This motivates us to build a more reliable noise detector in multimodal learning. To this end, we theoretically analyze the relationship between representation structure and noise detection capability. Based on this analysis, we propose REFINE, which is a multimodal label-noise detection framework that jointly uses fused and unimodal representations for label-noise detection. Specifically, REFINE constructs discriminative eigenvectors through discriminative analysis of the target and background classes and selects trusted representation spaces with better noise detection capability for each class. Within each trusted space, REFINE measures the alignment between each instance representation and the discriminative eigenvectors. It then combines the subsets selected from these spaces. The combined set provides cleaner supervision for updating the multimodal classifier, thereby reducing the influence of mislabeled examples during training and improving model generalization. Extensive experiments across diverse tasks demonstrate REFINE's superiority compared to baseline methods. The source code will be publicly available.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.11057 [cs.CV] |
| (or arXiv:2610.11057v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.11057 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xiaobo Xia [view email]
[v1]
Thu, 8 Oct 2026 01:14:41 UTC (1,999 KB)
来源：arXiv：cs.LG · arxiv.org