arXiv：cs.AI· Mingwei Fu, Zheshu Song, Zhikai Zhou, Shen Huang, Shidong Shang, Mengxiao Zhu·· 16 小时前
ScreenASR-Bench：带视觉语境的分层语音识别基准发布
ScreenASR-Bench: A Tiered Benchmark for Speech Recognition with Visual Context
AI 导读
ScreenASR-Bench 是一个分三个难度层级的语音识别基准，分别考察屏幕直接文本证据（ETE）、可见内容视觉证据（EVE）和需场景级推断的隐式语义证据（ISE）。
正文
Abstract:Current ASR models perform well on general benchmarks but still struggle with acoustically ambiguous words. Visual context can aid recognition in screen-based scenarios, yet existing benchmarks provide limited insight into the benefits of different types of visual evidence. To address this gap, we introduce ScreenASR-Bench , a tiered benchmark with three progressively challenging levels: Explicit Textual Evidence (ETE), based on direct on-screen text; Explicit Visual Evidence (EVE), using visible content rather than on-screen text; and Implicit Semantic Evidence (ISE), requiring scene-level inference. Extensive experiments on ASR models, Large Audio-Language Models (LALMs) and Omni models show that incorporating visual context substantially improves recall over the audio-only setting across all three model families. The gains are largest for ETE and smallest for ISE, indicating that models leverage direct visual evidence more effectively than indirect semantic cues. To facilitate future research on visually grounded speech recognition, we publicly release ScreenASR-Bench.
| Comments: | 5 pages |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.16260 [cs.AI] |
| (or arXiv:2610.16260v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.16260 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zheshu Song [view email]
[v1]
Mon, 12 Oct 2026 06:36:20 UTC (5,228 KB)
来源：arXiv：cs.AI · arxiv.org