基础模型能否识别视频高光？多模态目标基准研究

View PDF HTML (experimental)

Abstract:Foundation models are used for many real-world applications involving language understanding and generation from temporally-ordered multimodal events. In this work, we study the ability of models to identify the most important sub-events in a video, which is a fundamental prerequisite for summarizing, reasoning over, or narrating multimodal events. Specifically, we focus on football games and evaluate models on their ability to distinguish between important and non-important sub-events in a game. To this end, we construct a new dataset by leveraging publicly available football highlight reels containing the most important moments of a game according to professional experts, without any additional annotation costs. Using our dataset, we compare several state-of-the-art multimodal models and show that they fail to reliably capture contextual importance. Analyses of models beyond standard evaluation metrics reveal their tendency to rely on a single dominant modality and their ineffectiveness in synthesizing necessary information from multiple sources. Our findings underline the importance of modular architectures that dynamically adapt to multimodal heterogeneity and the need for complementary training procedures that maximize cross-modal synergy.

Subjects:	Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
Cite as:	arXiv:2601.16333 [cs.CV]
	(or arXiv:2601.16333v3 [cs.CV] for this version)
	https://doi.org/10.48550/arXiv.2601.16333 arXiv-issued DOI via DataCite

Submission history

From: Aditya K Surikuchi [view email]
[v1] Thu, 22 Jan 2026 21:40:08 UTC (2,294 KB)
[v2] Thu, 5 Mar 2026 09:51:43 UTC (2,283 KB)
[v3] Tue, 13 Oct 2026 13:11:47 UTC (2,559 KB)