arXiv：cs.CL· Patrick Schrottenbacher, Leon Hammerla, Lydia Kleine, Doris Stingl, Alexander Mehler·· 3 小时前AI 评分32
InterView-C：VR 虚拟形象访谈的同步多模态语料库
InterView-C: A Synchronized Multimodal Corpus of VR Avatar-Mediated Survey Interviews
AI 导读
InterView-C 是一个德语多模态语料库，收录 27 场完全在 VR 中进行、双方均以虚拟形象出镜的问卷调查访谈，将语音交互与注视、头部与身体运动、面部行为、手部及手指追踪数据同步对齐。
正文
Abstract:We present InterView-C, a German multimodal corpus of 27 survey interviews conducted entirely in virtual reality, with both interlocutors represented by avatars. The corpus aligns spoken interaction with synchronized behavioral data, including gaze, head and body movement, facial behavior, hand and finger tracking. Its reference transcripts and linguistic annotations provide a reliable interface between this multimodal spoken interaction and predominantly text-based NLP methods. This interface is important because automatically transcribing speech can distort linguistically relevant information, while downstream models trained on existing resources may additionally face transfer challenges when applied to transcribed spoken data. InterView-C therefore provides word-timed and manually post-edited verbatim transcripts for all 54 recordings, interview-item timings, questionnaire responses and negation cue and scope annotations for 1,422 sentences, 1,398 of them doubly annotated ({\alpha}=0.87 for cues; {\alpha}=0.81 for scopes). We demonstrate both challenges empirically: nine open-weight ASR systems disproportionately misrecognize short closed answers and number words, while negation models trained on existing corpora show lower and highly variable performance on our transcribed interviews than a model trained on the InterView-C annotations. InterView-C thus enables linguistic analyses of spoken interaction while retaining their alignment with rich multimodal behavior.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.10145 [cs.CL] |
| (or arXiv:2610.10145v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.10145 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Leon Hammerla [view email]
[v1]
Wed, 7 Oct 2026 14:23:32 UTC (1,757 KB)
来源：arXiv：cs.CL · arxiv.org