arXiv：cs.LG（机器学习，全量分类）· Nicholas Roberts, Xintong Li, Dyah Adila, Sonia Cromp, Tzu-Heng Huang, Jitian Zhao, Frederic Sala·· 15 小时前AI 评分33
几何感知适配：无需重训即可扩展预训练模型至新类别
Geometry-Aware Adaptation for Pretrained Models
AI 导读
研究者提出 Loki，一种用 Fréchet 均值替换 argmax 的即插即用预测规则，无需额外训练即可让预训练模型预测新类别或提升零样本性能。在 ImageNet 上，借助外部度量，Loki 相对 SimCLR 最高提升 29.7%，并可扩展至数十万类别；无外部度量时，用类别嵌入自导度量在 CLIP 等零样本模型上提升 10.5%。
正文
Abstract:Machine learning models -- including prominent zero-shot models -- are often trained on datasets whose labels are only a small proportion of a larger label space. Such spaces are commonly equipped with a metric that relates the labels via distances between them. We propose a simple approach to exploit this information to adapt the trained model to reliably predict new classes -- or, in the case of zero-shot prediction, to improve its performance -- without any additional training. Our technique is a drop-in replacement of the standard prediction rule, swapping argmax with the Fréchet mean. We provide a comprehensive theoretical analysis for this approach, studying (i) learning-theoretic results trading off label space diameter, sample complexity, and model dimension, (ii) characterizations of the full range of scenarios in which it is possible to predict any unobserved class, and (iii) an optimal active learning-like next class selection procedure to obtain optimal training classes for when it is not possible to predict the entire range of unobserved classes. Empirically, using easily-available external metrics, our proposed approach, Loki, gains up to 29.7% relative improvement over SimCLR on ImageNet and scales to hundreds of thousands of classes. When no such metric is available, Loki can use self-derived metrics from class embeddings and obtains a 10.5% improvement on pretrained zero-shot models such as CLIP.
| Comments: | NeurIPS 2023 |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML) |
| Cite as: | arXiv:2307.12226 [cs.LG] |
| (or arXiv:2307.12226v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2307.12226 arXiv-issued DOI via DataCite |
Submission history
From: Nicholas Roberts [view email]
[v1]
Sun, 23 Jul 2023 04:48:41 UTC (5,803 KB)
[v2]
Tue, 28 Nov 2023 04:35:51 UTC (6,052 KB)
[v3]
Thu, 1 Oct 2026 13:43:50 UTC (3,246 KB)
来源：arXiv：cs.LG（机器学习，全量分类） · arxiv.org