EduBehaviors：基于断言的教育对话可审计标注框架

View PDF HTML (experimental)

Abstract:Large language models have enabled rapid deployment of language classification and analysis, allowing users to describe complex constructs in natural language, such as pedagogical moves or expressions of learning and engagement, and label them in conversational datasets such as classroom or tutoring transcripts. However, because LLM reasoning is opaque, we have no verifiable, mechanistic insight into why a model chose a label for an utterance, and output differences over time or across models challenge replication. We introduce the EduBehaviors framework to generate interpretable, reliable, and scalable construct annotations in educational dialogue data that retain human agency while leveraging the strengths of generative AI, encoders, and machine learning models. We evaluate the framework on the TalkMoves dataset, predicting the Teacher TalkMoves labels. Our best configuration results in a macro-F1 of 0.764 and 0.756 Cohen's kappa, demonstrating improvements over direct prompting approaches and showing competitiveness with fine-tuned classifiers while gaining the benefit of greater transparency and comparability. To facilitate adoption and operationalization of the EduBehaviors framework in educational dialogue data by other researchers, we also release the EduBehaviors Toolkit with two complementary open-source tools based on the TalkMoves application.

Subjects:	Computation and Language (cs.CL); Computers and Society (cs.CY); Machine Learning (cs.LG)
Cite as:	arXiv:2609.27043 [cs.CL]
	(or arXiv:2609.27043v2 [cs.CL] for this version)
	https://doi.org/10.48550/arXiv.2609.27043 arXiv-issued DOI via DataCite

Submission history

From: Ana Trindade Ribeiro [view email]
[v1] Tue, 22 Sep 2026 20:38:54 UTC (869 KB)
[v2] Tue, 13 Oct 2026 22:50:27 UTC (3,852 KB)