arXiv：cs.CL· Yusong Wang, Keyu Mao, Takao Obi, Minghao Shao, Kotaro Funakoshi·· 6 小时前AI 评分29
手语对话中的情感识别：eJSL Dialog 数据集与基准测试
Emotion Recognition in Sign Language Conversation
AI 导读
研究者将对话情感识别（ERC）任务引入手语视频分析，并提出基于 STUDIES 语料库脚本构建的 eJSL Dialog 数据集，包含 1,920 个视频样本、480 组对话。团队用从孤立视觉网络到多模态对话架构的模型进行系统基准测试，结果显示在现有基准设置下将对话式 ERC 框架扩展到手语对话可行，但现有视觉表征在捕捉手语特有的情感线索上仍存在局限。
正文
Abstract:Emotion Recognition in Conversation is a core component of affective computing, while current sign language emotion datasets primarily focus on isolated sentences and lack conversational context. Models trained exclusively on these isolated utterances demonstrate degraded performance in real world scenarios because they cannot utilize historical dialogue flow. To address this structural limitation, we introduce the ERC task to sign language video analysis and propose the eJSL Dialog dataset. Constructed using the scripts from the STUDIES corpus, the dataset contains 1,920 video samples organized into 480 unique dialogues. We conduct systematic benchmarking on this dataset using models ranging from isolated visual networks to multimodal conversational architectures. The results suggest the feasibility of extending conversational ERC frameworks to sign-language dialogue under the current benchmark setting, while also revealing limitations in existing visual representations for capturing sign-specific affective cues, motivating future work on sign-specific visual modeling and larger sign-language conversational training resources.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2605.23328 [cs.CL] |
| (or arXiv:2605.23328v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2605.23328 arXiv-issued DOI via DataCite |
Submission history
From: Yusong Wang [view email]
[v1]
Fri, 22 May 2026 07:44:20 UTC (2,585 KB)
[v2]
Mon, 13 Jul 2026 13:17:59 UTC (2,585 KB)
[v3]
Tue, 6 Oct 2026 04:43:20 UTC (2,588 KB)
来源：arXiv：cs.CL · arxiv.org