arXiv：cs.AI· Claire Schlesinger, Wolfgang Gatterbauer, Robin Walters·· 10 小时前
如何度量模型学到的等变性：一种信息论方法
Measuring Learned Equivariance: An Information Theoretic Approach
AI 导读
研究提出用归一化互信息（NMI）检验学习到的等变性：在有限离散设定下，等变性要求原始输入与变换输入的表示能互相精确确定。针对高维激活，采用 centered kernel alignment（CKA）作为可扩展检验，并对正交特征变换给出精确保证。跨架构与任务实验显示，数据增强通常提升变换一致性，表明可直接从表示探测学到的等变性，无需还原隐藏的变换规则。
正文
Abstract:Learning symmetries through data augmentation offers an alternative to architectural equivariance, but does not guarantee that a network respects those symmetries. Measuring learned equivariance is especially difficult for intermediate representations, whose group actions are not specified. In a finite discrete setting, we show that equivariance requires representations of original and transformed inputs to determine one another exactly, yielding a test based on normalized mutual information (NMI). For high-dimensional activations, we use centered kernel alignment (CKA) as a scalable test, with an exact guarantee for orthogonal feature transformations. Across architectures and tasks, augmentation usually increases transformation consistency, showing that learned equivariance can be probed directly from representations without recovering the hidden transformation rules that relate them.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Information Theory (cs.IT) |
| Cite as: | arXiv:2610.17269 [cs.LG] |
| (or arXiv:2610.17269v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.17269 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Claire Schlesinger [view email]
[v1]
Mon, 12 Oct 2026 18:24:19 UTC (2,815 KB)
来源：arXiv：cs.AI · arxiv.org