arXiv：cs.CL· Zhifan Sun, Sebastian Gombert, Jannik Lossjew, Tobias Wyrwich, Berrit Katharina Czinczel, David Bednorz, Marcus Kubsch, Knut Neumann, Hendrik Drachsler·· 3 小时前AI 评分37
Alice：面向评分量规的多维自动简答评分大规模德语基准
Alice: A Large-Scale German Benchmark for Rubric-Based Multi-Dimensional Automatic Short Answer Scoring
AI 导读
研究者发布 Alice，一个基于评分量规的大规模德语自动简答评分（ASAS）数据集，包含学习表现（Alice-LP）、知识元素（Alice-KE）和技能（Alice-SK）三个子任务，并将量规式 ASAS 形式化为量规检索任务。
正文
Abstract:Automatic Short Answer Scoring (ASAS) is central to NLP for Education. However, openly available benchmarks remain scarce, and existing datasets largely address how well students answer a question directly rather than how well they master underlying concepts (knowledge elements) such as thermal energy or epistemic activities (skills) such as reasoning or claim.
To address this gap, we introduce Alice, a large-scale, rubric-based German ASAS dataset that is pedagogically aligned and comprises three subtasks: (i) learning performance (Alice-LP), (ii) knowledge elements (Alice-KE), and (iii) skills (Alice-SK).
We further formulate rubric-based ASAS as a rubric-retrieval task and benchmark the dataset with a range of language models, from encoder-only models to lightweight LLMs. We also benchmark the dataset with zero-shot prompting via LLMs and a standard classification baseline. The experiments show that LLMs, in particular, struggle to score knowledge elements and skills in the zero-shot setting. They also indicate that rubric text is often useful, especially for Alice-KE and Alice-SK, while on Alice-LP gains over sample-solution-focused inputs are more modest and vary by model and input format.
| Comments: | EMNLP2026 Main |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.09661 [cs.CL] |
| (or arXiv:2610.09661v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.09661 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhifan Sun [view email]
[v1]
Wed, 7 Oct 2026 08:29:43 UTC (526 KB)
来源：arXiv：cs.CL · arxiv.org