arXiv：cs.CL· Yihang Li, Chenhui Chu·· 3 小时前AI 评分35
重新思考会议有效性：时序细粒度自动会议有效性评估的基准与框架
Rethinking Meeting Effectiveness: A Benchmark and Framework for Temporal Fine-grained Automatic Meeting Effectiveness Evaluation
AI 导读
研究者提出以"目标达成速率随时间变化"来定义会议有效性，并对会议内各主题片段逐一评估。为此构建了 AMI-ME 数据集，包含来自 130 场 AMI Corpus 会议、2,459 个人工标注片段，同时开发了以 LLM 作为评判者的自动评估框架，并建立覆盖商业场景到非结构化讨论的基准，还从原始语音起做了端到端评测。数据集与代码将公开。
正文
Abstract:Evaluating meeting effectiveness is crucial for improving organizational productivity. Current approaches rely on post-hoc surveys that yield a single coarse-grained score for an entire meeting. The reliance on manual assessment is inherently limited in scalability, cost, and reproducibility. Moreover, a single score fails to capture the dynamic nature of collaborative discussions. We propose a new paradigm for evaluating meeting effectiveness centered on novel criteria and temporal fine-grained approach. We define effectiveness as the rate of objective achievement over time and assess it for individual topical segments within a meeting. To support this task, we introduce the AMI Meeting Effectiveness (AMI-ME) dataset, a new meta-evaluation dataset containing 2,459 human-annotated segments from 130 AMI Corpus meetings. We also develop an automatic effectiveness evaluation framework that uses a Large Language Model (LLM) as a judge to score each segment's effectiveness relative to the overall meeting objectives. Through substantial experiments, we establish a comprehensive benchmark for this new task and evaluate the framework's generalizability across distinct meeting types, ranging from business scenarios to unstructured discussions. Furthermore, we benchmark end-to-end performance starting from raw speech to measure the capabilities of a complete system. Our results validate the framework's effectiveness and provide strong baselines to facilitate future research in meeting analysis and multi-party dialogue. Our dataset and code will be publicly available. The AMI-ME dataset and the Automatic Evaluation Framework are available at this https URL.
| Comments: | ACL 2026 Main Conference |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2604.17260 [cs.CL] |
| (or arXiv:2604.17260v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2604.17260 arXiv-issued DOI via DataCite |
Submission history
From: Yihang Li [view email]
[v1]
Sun, 19 Apr 2026 04:59:53 UTC (544 KB)
[v2]
Thu, 4 Jun 2026 05:26:22 UTC (544 KB)
[v3]
Wed, 7 Oct 2026 05:20:44 UTC (555 KB)
来源：arXiv：cs.CL · arxiv.org