arXiv：cs.LG· Saumyaranjan Mohanty, Aravind Reddy, Konda Reddy Mopuri·· 13 小时前
X-AVDD：基于交叉注意力的音视频数据集蒸馏方法
X-AVDD: Cross-Attentive Audio-Visual Dataset Distillation
AI 导读
X-AVDD 是一种基于交叉注意力的音视频数据集蒸馏方法，通过将跨模态对齐与联合优化解耦，端到端合成成本降低约 8 倍 GPU-hours。在 VGGSound 上，SPC=10 时 top-1 测试准确率从 8.2% 提升至 24.0%，SPC=20 时从 9.8% 提升至 26.5%。
正文
Abstract:Audio-visual dataset distillation aims to synthesize a small dataset from a large paired audio-visual dataset, while nearly preserving the training utility for neural networks. Existing approaches such as DM and AVDD face two critical limitations: 1) they are memory-intensive, making them difficult to scale to complex architectures and higher samples-per-class (SPC) settings, and 2) they often fail to preserve strong cross-modal semantic correspondence. To tackle these issues, we propose X-AVDD, a cross-attention-based distillation method that explicitly captures inter-modal relationships during synthesis. By decoupling cross-modal alignment from joint optimization, X-AVDD is substantially more memory-efficient, reducing end-to-end synthesis cost by $\sim 8\times$ GPU-hours, while producing highly aligned synthetic data. Empirically, X-AVDD significantly outperforms other audio-visual distillation methods. On VGGSound, an audio-visual dataset of short YouTube clips, X-AVDD improves top-1 test accuracy from $8.2\%\rightarrow24.0\%$ at SPC=$10$ and $9.8\%\rightarrow26.5\%$ at SPC=$20$. Furthermore, X-AVDD improves cross-architecture generalization (e.g., $25.5\%\rightarrow35.3\%$ for a ViT target on AVE dataset at SPC $=50$) and successfully scales to complex architectures such as AudioCLIP, achieving $62.3\%$ accuracy using only $5.65\%$ of the full dataset, against the full dataset accuracy of $72.4\%$ while completing in $9\times$ lower training time compared to the full dataset.
| Comments: | Accepted at NeurIPS 2026. Camera-ready version forthcoming. 17 pages, 6 figures, 16 tables |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Sound (cs.SD) |
| Cite as: | arXiv:2610.13687 [cs.CV] |
| (or arXiv:2610.13687v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.13687 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Aravind Reddy [view email]
[v1]
Fri, 9 Oct 2026 14:51:26 UTC (1,907 KB)
来源：arXiv：cs.LG · arxiv.org