arXiv：cs.AI· Jialin Wu, Meenakshi Khosla·· 16 小时前
大语言模型表征对齐的尺度、驱动因素与局限
Scale, Drivers, and Limits of Representational Alignment Across Large Language Models
AI 导读
研究提出 Neighborhood-Weighted RSA（NW-RSA）与 Counterfactual RSA（CF-RSA）两种方法，用于定位大语言模型在哪些几何尺度上表征一致、以及哪些语言属性支撑这种一致。结果表明，不同语言模型在粗粒度表征几何上高度一致，但在局部邻域存在显著分歧；逻辑辖域变化对对齐的削弱明显大于层级句法、事件角色或篇章结构的变化。
正文
Abstract:Comparing large language models can reveal shared representational structure and systematic differences between them. However, aggregate similarity measures can obscure how agreement varies across geometric scales and which linguistic properties support it. We introduce two complementary methods to resolve where models agree and what supports that agreement. Standard representational similarity analysis (RSA) summarizes similarity across all stimulus pairs, potentially obscuring differences in alignment between nearby and distant inputs. Neighborhood-Weighted RSA (NW-RSA) addresses this limitation by weighting stimulus pairs according to their proximity in representational space, placing alignment from local neighborhoods to global geometry on a common scale. Counterfactual RSA (CF-RSA) tests which linguistic properties support alignment by comparing intact representations in one model with representations under controlled linguistic perturbations in another. Across diverse language models, we find strong agreement in coarse representational geometry but substantial divergence in local neighborhoods. Alignment is observed both across broadly sampled natural text and within linguistic equivalence classes that preserve particular aspects of meaning or structure. Counterfactual analyses reveal selective sensitivity: changes to logical scope reduce alignment substantially more than the tested changes to hierarchical syntax, event roles, or discourse structure. Alignment also decreases when sentence structure or lexical meaning is disrupted in word lists and jabberwocky sentences, suggesting that the regularities of natural language support stronger convergence. These findings refine claims of representational universality by showing that strong overall similarity need not imply agreement in how models represent finer linguistic distinctions.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.14476 [cs.CL] |
| (or arXiv:2610.14476v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.14476 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jialin Wu [view email]
[v1]
Sat, 10 Oct 2026 02:12:32 UTC (1,167 KB)
来源：arXiv：cs.AI · arxiv.org