arXiv：cs.CL· Sihan Ren, Gaozheng Li, Yuanshang Quan, Yiming Qin, Fuyi Yang, Chang Liu, Lan Xu, Minye Wu·· 4 小时前AI 评分36
ReSCUE：面向未分段长视频流式手语翻译的句子提交重译框架
ReSCUE: Re-translation with Sentence Commitment for Unsegmented Long-Form Simultaneous Sign Language Translation
AI 导读
ReSCUE 是一个面向未分段长视频的流式手语翻译统一框架，通过推理感知训练、稳定重译和句子提交机制，在训练与推理阶段对齐真实流式条件。在标准句子级基准上，它在低延迟设置下实现更低延迟和最佳翻译质量；在长视频未分段数据集上，其翻译质量接近使用真实句子边界的 oracle 离线系统，同时延迟大幅降低。该工作已被 NeurIPS 2026 接收。
正文
Abstract:Simultaneous Sign Language Translation (SLT) is critical for real-time communication, yet existing methods remain largely confined to sentence-level, offline settings that assume pre-segmented inputs. These assumptions hinder deployment in realistic scenarios involving continuous, unsegmented video streams. We present ReSCUE, a unified framework for simultaneous SLT on unsegmented long-form sign language videos that aligns training and inference with realistic streaming conditions. ReSCUE combines inference-aware training to handle partial inputs, non-signing pauses, and multi-sentence contexts, stabilized re-translation to enable low-latency yet revisable predictions with reduced output flicker, and a sentence commitment mechanism for online segmentation and memory management. Experiments on standard sentence-level benchmarks show that ReSCUE achieves lower latency and the best translation quality under low-latency settings. On long-form unsegmented datasets, ReSCUE approaches the translation quality of oracle offline systems that use ground-truth sentence boundaries, while operating at substantially lower latency, demonstrating its practicality for real-world streaming scenarios.
| Comments: | Accepted at NeurIPS 2026 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.03022 [cs.CV] |
| (or arXiv:2610.03022v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.03022 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Sihan Ren [view email]
[v1]
Fri, 2 Oct 2026 08:58:00 UTC (4,793 KB)
来源：arXiv：cs.CL · arxiv.org