Auto-AEG：面向开放词汇音频事件定位的可扩展数据构建方法
Auto-AEG: Scalable Data Construction for Open-Vocabulary Audio Event Grounding
Auto-AEG 通过自动数据构建与模型微调，为开放词汇音频事件定位生成时序监督数据，结合程序合成带精确区间的片段做冷启动与真实音频上的多模型伪标签强化学习。该方法在独立基准 AEGBench 上大幅提升大音频语言模型的时序定位能力，并泛化至多个 SED 与音频定位基准。
Authors:Zihan Zhang, Xize Cheng, Wenhao Yan, Tong Zhang, Dongjie Fu, Wenxu Jia, Boyun Zhang, Yongbo He, Qifei Zhang, Linjun Li, Wenshi Chen, Tao Jin
Abstract:Large Audio-Language Models (LALMs) reason fluently about sound yet struggle to localize precisely when events occur, while classical Sound Event Detection (SED) attains frame-level precision only over a closed label set. At the intersection of these paradigms lies the task of Open-Vocabulary Audio Event Grounding: predicting all time intervals of a target sound event described by an arbitrary natural language query. Progress is bottlenecked by data scarcity: no large-scale resource provides open-vocabulary onset/offset supervision, and manual temporal annotation is prohibitively expensive. To address this, we introduce Auto-AEG, a scalable pipeline that constructs such supervision by automatic data construction and model fine-tuning. It pairs programmatically synthesized clips, which carry placement-exact ground-truth intervals for supervised cold-start, with multi-model pseudo-labels on real-world audio that supply the reward signal for reinforcement learning. Training with this pipeline yields large temporal-localization gains on AEGBench, an independent difficulty-stratified benchmark whose annotations we release, and these gains generalize to various held-out SED and audio grounding benchmarks. Our results show that automatically constructed data, coupled with interval-aware reward design, provides an effective data-side route to expanding the temporal localization capability of LALMs.
| Comments: | Work in progress |
| Subjects: | Sound (cs.SD); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.04383 [cs.SD] |
| (or arXiv:2607.04383v5 [cs.SD] for this version) | |
| https://doi.org/10.48550/arXiv.2607.04383 arXiv-issued DOI via DataCite |
Submission history
From: Zihan Zhang [view email]
[v1]
Sun, 5 Jul 2026 16:22:14 UTC (1,478 KB)
[v2]
Tue, 21 Jul 2026 03:31:22 UTC (1,487 KB)
[v3]
Thu, 23 Jul 2026 08:55:09 UTC (1,398 KB)
[v4]
Wed, 29 Jul 2026 12:04:35 UTC (1,404 KB)
[v5]
Tue, 13 Oct 2026 07:23:33 UTC (1,459 KB)
来源：arXiv：cs.AI · arxiv.org