arXiv：cs.AI· Jiawei Xiong, Cheng Tang, Hong Jiao, Hanna Choi·· 4 小时前
用证据中心设计（ECD）指导测量中的提示工程：以自动作文评分为例
Using Evidence-Centered Design to Guide Prompt Engineering in Measurement: An Illustration with Automated Essay Scoring
AI 导读
研究提出用教育测量中的证据中心设计（ECD）框架替代当前试错式的提示工程实践，将学生模型、证据模型和任务模型映射到教育提示设计组件，并据此推导出五条设计原则。在自动评分任务上，ECD 设计的提示词与 SOTA 提示基线进行了实证对比，结果显示将提示视为评估设计而非非正式指令，可提升 AI 生成教育输出的效度、信度与公平性。
正文
Abstract:Generative artificial intelligence (GenAI) models, particularly large language models (LLMs), are increasingly used in educational settings for tasks such as item generation, automated scoring, and feedback delivery. The quality of these outputs depends not only on the AI model but also on how users construct the prompts that guide model behaviors. Current prompt engineering practices are largely ad hoc, relying on trial and error rather than principled design. This lack of structure introduces validity threats including construct misrepresentation, construct-irrelevant variance, and limited reproducibility. We propose that Evidence-Centered Design (ECD), a framework in educational measurement for structuring valid inferences from observed performance, can serve as a principled foundation for prompt engineering in educational applications. We map the three core ECD models, the student model, the evidence model, and the task model, onto the components of educational prompt design, and describe a workflow for ECD-based prompt engineering. We derive five design principles from this mapping and present an empirical study contrasting ECD-designed prompts with state-of-the-art (SOTA) prompting baselines on an automated scoring task. Our empirical study demonstrates the usefulness of this framework on automated scoring. Results suggest that treating prompting as assessment design rather than informal instructions strengthens the validity, reliability, and fairness of AI-generated educational outputs.
| Subjects: | Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC) |
| Cite as: | arXiv:2610.15231 [cs.AI] |
| (or arXiv:2610.15231v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.15231 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Hong Jiao [view email]
[v1]
Sat, 10 Oct 2026 23:51:48 UTC (1,669 KB)
来源：arXiv：cs.AI · arxiv.org