arXiv：cs.AI· Julien Kordes, Reuben Dorent·· 11 小时前
仅需百对配对数据：基于蒸馏最优传输的表征对齐方法
Representation Alignment from a Few Paired Data via Distilled Optimal Transport
AI 导读
一种新方法在仅 N ~ 100–500 对配对样本、两侧无配对数据充足的情况下对齐两个独立预训练的单模态编码器。它用与 N 个配对锚点的相似度将两模态放入同一坐标系，以单个熵最优传输问题匹配两个点云，并将传输方案蒸馏为两个轻量编码器。在 Flickr30k、MS-COCO 和 ChEBI-20 的图像-文本与分子-文本对齐任务上，相同锚点预算下优于现有方法。
正文
Abstract:Aligning two independently pretrained unimodal encoders typically demands millions of paired examples. In this paper, we study the regime where only $N \sim 100$--$500$ pairs are available, while unpaired data on either side is abundant. Our method expresses every representation by its similarities to the $N$ paired anchors, which places both modalities in a common coordinate system; matches the two point clouds with a single entropic optimal transport problem solved over the entire pool, whose marginal constraints counteract the hubness that defeats nearest-neighbor matching; and distills the resulting plan into two lightweight encoders, so that the correspondence becomes inductive. We additionally put this marginal constraint in relation with the one-step hub corrections of the bilingual lexicon induction literature. Pairing is required only for the anchors: the unpaired collections may be drawn independently from the two marginals, so that no item outside the anchor set needs to have both of its modalities observed. We evaluate on image-text and molecule-text alignment, where our method outperforms existing approaches at the same anchor budget on Flickr30k, MS-COCO and ChEBI-20.
| Comments: | Preprint |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.12781 [cs.AI] |
| (or arXiv:2610.12781v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.12781 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Reuben Dorent [view email]
[v1]
Thu, 8 Oct 2026 22:38:46 UTC (13,412 KB)
来源：arXiv：cs.AI · arxiv.org