arXiv：cs.AI· Yu Pan, Guangzhao Yang, Muhammad Huzaifah, Ningjie Bai, Shi Qiu·· 5 小时前
JAL-Turn：面向全双工语音对话的联合声学-语言建模实时轮次检测
JAL-Turn: Joint Acoustic-Linguistic Modeling for Real-Time and Robust Turn-Taking Detection in Full-Duplex Spoken Dialogue Systems
AI 导读
JAL-Turn 是一个轻量级纯语音轮次检测框架，通过跨注意力模块自适应融合预训练声学与语言表征，预测 hold 或 shift，并共享冻结的 ASR 编码器，使轮次预测与语音识别完全并行、几乎不增加端到端延迟。
正文
Abstract:Efficient and robust turn-taking detection remains a significant challenge in industrial-grade voice AI agents. Existing systems often rely on acoustic or linguistic cues alone, limiting accuracy and stability, whereas directly endowing large language models with full-duplex capability incurs heavy data and deployment costs that hinder real-time usage. We propose JAL-Turn, a lightweight speech-only turn-taking framework based on a joint acoustic-linguistic modeling paradigm, in which a cross-attention module adaptively fuses pre-trained acoustic and linguistic representations for low-latency hold or shift prediction. By sharing a frozen ASR encoder, JAL-Turn enables turn-taking prediction to run fully in parallel with speech recognition, introducing almost no additional end-to-end latency. We further introduce a scalable pipeline that automatically derives reliable turn-taking labels from large-scale real-world dialogue corpora. Experiments on the public Easy-Turn and Smart-Turn benchmarks and an in-house Japanese customer-service dataset show that JAL-Turn achieves competing results among strong baselines in detection accuracy while maintaining superior real-time performance.
| Comments: | Accepted to NCMMSC 2026 for oral presentation |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2603.26515 [cs.CL] |
| (or arXiv:2603.26515v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2603.26515 arXiv-issued DOI via DataCite |
Submission history
From: Yu Pan [view email]
[v1]
Fri, 27 Mar 2026 15:25:38 UTC (1,136 KB)
[v2]
Mon, 12 Oct 2026 13:14:32 UTC (937 KB)
来源：arXiv：cs.AI · arxiv.org