arXiv：cs.AI· Mohammed Shabot, Jonathan Adams, Jan Van den Stock, Joost Vennekens·· 6 小时前
用大语言模型预筛视频元数据，降低视频数据集构建成本
Towards Cost-Efficient Video Dataset Construction with Large Language Models
AI 导读
研究提出用 LLM 基于元数据预筛视频，在下载前完成相关性判断，以节省存储、时间和标注成本。LLM 预筛与两种关键词启发式基线对比，并以四名独立标注者的严格多数投票为参照，通过 Gwet's AC1 和 LOO、AORI 机制评估其作为虚拟标注者的表现，最后给出成本对比。实现已在 GitHub 开源。
正文
Abstract:Research on video classification models has witnessed significant growth recently, accompanied by a significant increase in the demand for datasets to train such models. Existing video dataset curation approaches share a common bottleneck: candidate videos must be downloaded before any relevance judgment is made, which results in storage, time and annotation costs. In this work, we present a filtering step, using Large Language Models (LLMs) to pre-screen videos based on their metadata, before actually downloading these videos, thereby saving on these costs. We evaluate the LLMs against a strict human majority vote derived from four independent annotators and validate the reliability of the annotations through inter-annotator agreement analysis (Gwet's AC1). We compare the LLM pre-filter against two no-LLM baselines, namely two keyword heuristics. Via a Leave-One-Rater-Out (LOO) mechanism and an Add-One-Rater-In (AORI), we evaluate the performance of the LLMs as virtual annotators. Finally, we report cost comparisons relative to existing approaches. To ensure reproducability, we made our implementation available on github.
| Comments: | 18 pages, 9 tables, 1 figure |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.16857 [cs.CV] |
| (or arXiv:2610.16857v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.16857 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Mohammed Shabot [view email]
[v1]
Mon, 12 Oct 2026 14:26:44 UTC (6,286 KB)
来源：arXiv：cs.AI · arxiv.org