arXiv：cs.CL· Emily \"Ohman, Anna Koufakou·· 11 小时前
同名情绪标签并非同一测量目标：GoEmotions、XED 与 BRIGHTER 的跨数据集标签等价性研究
Fear Is Not Always Fear: Variation in Emotion Label Equivalence Across Datasets
AI 导读
研究检验 GoEmotions、XED、BRIGHTER 共有的五个情绪标签是否含义一致，发现同一批文本上两个 fear 分类器分别标记 71% 和 8% 为恐惧，但排序高度一致。
正文
Abstract:Emotion classifiers are routinely applied beyond their training data, on the assumption that same-named labels such as fear or surprise mean the same thing across resources. We test this assumption for five labels shared by GoEmotions, XED, and BRIGHTER, comparing models on identical inputs and under cross-corpus transfer. On the same texts, one fear classifier flags 71% as fearful and another 8%, yet the two largely agree on how the texts rank. Across 30 resource-label comparisons, training-label prevalence explains about half of this kind of scale mismatch, and a fitted intercept removes most of it. What recalibration cannot repair is larger: transferred models lose 55% of native discrimination in aggregate, agreement in ranking between classifiers is only weakly related to that loss, and for surprise the resources disagree about which texts express the emotion. The choice of training resource can therefore change conclusions: on two external tweet collections, BRIGHTER- and XED-trained classifiers give opposite answers about which shows more surprise, even after rank normalization, with intervals excluding zero in both directions. A shared emotion label name does not establish an interchangeable measurement target.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.12841 [cs.CL] |
| (or arXiv:2610.12841v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.12841 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Emily Ohman [view email]
[v1]
Fri, 9 Oct 2026 00:55:54 UTC (476 KB)
来源：arXiv：cs.CL · arxiv.org