arXiv：cs.LG· Tobias Hallmen, Elisabeth Andr\'e·· 6 小时前AI 评分34
语言承载专家印象：基于评分工具的 LLM 评判器实现心理咨询质量跨域迁移并超越域内训练
Language Carries the Expert's Impression: Instrument-Anchored LLM Judges Transfer Counseling-Quality Assessment and Beat In-Domain Training
AI 导读
在三个德语模拟心理咨询语料（195 场专家评分会话）上，用其他领域数据训练 LLM 评判器预测专家总体印象，跨域迁移的嵌套 Spearman ρ 达 0.54，超过域内训练的 ≤0.48，会话级配对差距 +0.15，匹配训练集规模后仍为 +0.12。
正文
Abstract:Automatic assessment of communication quality in dyadic counseling conversations is bottlenecked by data: expert-rated corpora are small and expensive to grow. We study cross-domain transfer of expert overall-impression prediction across three German corpora of simulated counseling (two general-practice medical, one school-related parent-teacher; $n=195$ expert-rated sessions, one corpus after scale equating). Training on the other domains beats training in-domain: leave-one-domain-out transfer reaches nested Spearman $\rho = 0.54$ against $\le 0.48$ within the target domain, a paired session-level gap of $+0.15$ that holds at $+0.12$ when the training-set sizes are matched, so it is not simply data volume. The decisive features are session-level construct scores from small open-weight LLMs reading the two-speaker transcript, with the constructs largely derived from the experts' rating instruments: the instrument-derived battery lifts a single judge from $0.32$ to $0.41$ over generic dialogue qualities, judges from three model families ensemble to $0.51$ language-only, and a nonverbal-dyadic block adds $+0.03$ more, not separable from noise at this sample size. We also price the recording setup: one corpus lost its per-speaker audio, 16% of its diarised segments carry the wrong speaker, and repair is worth $+0.07$ there. At practically attainable corpus sizes, the expert's overall impression is carried by what is said, and by other communication programs' data more than by one's own.
| Comments: | Preprint. 25 pages, 2 figures |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.08055 [cs.CL] |
| (or arXiv:2610.08055v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.08055 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tobias Hallmen [view email]
[v1]
Tue, 6 Oct 2026 09:53:25 UTC (85 KB)
来源：arXiv：cs.LG · arxiv.org