arXiv：cs.AI· Vicente Balmaseda, Bokun Wang, Ching-Long Lin, Tianbao Yang·· 3 小时前
GloFND：自监督对比学习中全局假阴性动态发现方法
Discovering Global False Negatives On the Fly for Self-supervised Contrastive Learning
AI 导读
GloFND 提出一种基于优化的方法，在自监督对比学习训练过程中为每个锚点数据自动学习阈值，从而在全局数据集范围而非 mini-batch 局部识别假阴性样本。该方法每次迭代的计算开销与数据集规模无关，在图像和图文数据上的实验验证了其有效性。该工作已被 ICML 2025 接收，实现代码已开源。
正文
Abstract:In self-supervised contrastive learning, negative pairs are typically constructed using an anchor image and a sample drawn from the entire dataset, excluding the anchor. However, this approach can result in the creation of negative pairs with similar semantics, referred to as "false negatives", leading to their embeddings being falsely pushed apart. To address this issue, we introduce GloFND, an optimization-based approach that automatically learns on the fly the threshold for each anchor data to identify its false negatives during training. In contrast to previous methods for false negative discovery, our approach globally detects false negatives across the entire dataset rather than locally within the mini-batch. Moreover, its per-iteration computation cost remains independent of the dataset size. Experimental results on image and image-text data demonstrate the effectiveness of the proposed method. Our implementation is available at this https URL.
| Comments: | Accepted to ICML 2025 |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (stat.ML) |
| Cite as: | arXiv:2502.20612 [cs.LG] |
| (or arXiv:2502.20612v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2502.20612 arXiv-issued DOI via DataCite |
Submission history
From: Vicente Balmaseda [view email]
[v1]
Fri, 28 Feb 2025 00:28:25 UTC (6,794 KB)
[v2]
Wed, 25 Jun 2025 21:11:53 UTC (6,774 KB)
来源：arXiv：cs.AI · arxiv.org