arXiv：cs.CL· Sheza Munir, Ramaravind Kommiya Mothilal, Syed Ishtiaque Ahmed·· 11 小时前
AXLE：让标注者的亲身经历在数据标注中被激活
Activating the Annotator: Operationalising Lived Experience in Data Annotation
AI 导读
研究者提出 AXLE，一种预标注阶段：用场景小故事引出轴引导对话，合成为第一人称微叙事后再交给标注者开始打标。在涵盖移民、宗教与性别的模糊仇恨言论 2x2 被试间实验（104 名标注者、30 个条目、1,040 条标注）中，该方式带来更高的身份共鸣与伤害敏感度，同时提高标签熵、降低标注者间一致性，呈现多元而非制造共识。
正文
Abstract:Perspectivist NLP has established that annotator positionality shapes labels for subjective tasks, but the field still lacks methods for operationalising it. Positionality is typically operationalised as a demographic form completed at recruitment: a static attribute recorded once and treated as a covariate, on the assumption that the knowledge it indexes is active when the annotator reads an item. This paper treats that assumption as a design problem and asks whether lived experience can instead be primed by the task itself. We introduce AXLE, a pre-annotation stage in which a scenario vignette anchors a short, axis-guided conversation that is synthesised into a first-person micro-narrative and returned to the annotator before labelling begins. In a 2x2 between-subjects study (104 annotators, 30 items, 1,040 annotations) on ambiguous hate speech spanning immigration, religion and gender, elicitation was associated with higher identity resonance and higher sensitivity to harm. It also increased label entropy and lowered inter-annotator agreement: the stage appears to surface plurality rather than manufacture consensus, which distinguishes it from annotator training, whose success criterion is convergence.
| Subjects: | Computation and Language (cs.CL); Computers and Society (cs.CY); Human-Computer Interaction (cs.HC) |
| Cite as: | arXiv:2610.12701 [cs.CL] |
| (or arXiv:2610.12701v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.12701 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Sheza Munir [view email]
[v1]
Thu, 8 Oct 2026 20:31:56 UTC (566 KB)
来源：arXiv：cs.CL · arxiv.org