arXiv：cs.CL· Lin Ai, Emma Thronson, Katherine Conhaim, Amir Zeldes·· 4 小时前
PROPALIGN：基于多摘要的命题显著性分级评估方法与 PROPALIGNBANK 数据集
The Sum of All Summaries: PROPALIGN for Graded Proposition Salience Evaluation
AI 导读
PROPALIGN 是一种基于多摘要的命题显著性评分估计方法，并同步发布含 35K+ 命题、138K 对齐数据的 PROPALIGNBANK 数据集。研究覆盖英语多口语与书面体裁，发现直接让模型判断显著性会导致不一致，而基于摘要的分级指标更稳定；单模型表现不及人类，融合语言特征的模型集成则接近人工标注质量上限。
正文
Abstract:Identification and ranking of salient propositions in documents is indirectly involved in many AI tasks, but few papers define proposition salience scores clearly, make diverse data available, or evaluate models directly on the task. This paper proposes PROPALIGN, a multiple-summary-based approach for propositional salience score estimation, and releases PROPALIGNBANK, an accompanying dataset for the method with 138K alignments of over 35K propositions. We compare several approaches to salient proposition extraction in English across a broad range of spoken and written genres, and demonstrate that naively asking models for judgments leads to inconsistencies. By contrast, graded, summary-based metrics are more stable. We further observe that while single models perform worse than humans, an ensemble of models incorporating linguistic features approaches manual annotation quality ceilings. In addition, analyses of the model outputs suggest insights into how models can extract salient information with and without the use of summaries.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.19387 [cs.CL] |
| (or arXiv:2610.19387v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19387 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Lin Ai [view email]
[v1]
Wed, 14 Oct 2026 01:32:24 UTC (423 KB)
来源：arXiv：cs.CL · arxiv.org