arXiv：cs.CL· Noureddine Khaous (LLF - UMR7110, UPCit\'e), Guillaume Wisniewski (LLF - UMR7110, UPCit\'e)·· 13 小时前
研究：表征相似度指标比较什么比怎么比较更重要
A Better Metric Will Not Save You: What You Compare Beats How You Compare It
AI 导读
研究指出表征相似度指标（RSA、CKA、Procrustes）都默认条件集合给定，但该集合由“纳入哪些输入”和“如何分组”两项决定，任何不变性都无法触及。作者以十五种语言、二十七种表征的元音声学组织为基准，发现类别选择变化对两个表征间差距的影响超过所测效应本身的两倍，而类别数量变化几乎无影响。该设定下模型排名在特定选择下可识别，跨语言排名则不可识别。
正文
Abstract:Representational similarity measures (RSA, CKA, Procrustes) are chosen for the transformations they treat as irrelevant, and all of them take the set of conditions as given. That set is fixed by two decisions, which inputs are admitted and how they are grouped, both taken before the matrix a measure operates on exists: no invariance property reaches them. We measure what they are worth where the arrangement the conditions ought to have is known independently of the systems compared, the acoustic organisation of vowels, across fifteen languages and twenty-seven representations. Changing which categories are compared, at a fixed number of them, moves the gap between two representations by more than twice the effect being measured, while changing how many moves it not at all. In this setting rankings of models are identified under these choices, and rankings across languages are not.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.13476 [cs.CL] |
| (or arXiv:2610.13476v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.13476 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Guillaume Wisniewski [view email] [via CCSD proxy]
[v1]
Fri, 9 Oct 2026 12:11:53 UTC (101 KB)
来源：arXiv：cs.CL · arxiv.org