HuggingFace Daily Papers·· 1 天前
OmniDR-Bench：以视频为中心的深度研究报告智能体基准发布
Beyond Watching and Searching: Benchmarking Agents for Video-centered Deep Research Reports
AI 导读
研究提出以视频为中心的深度研究报告生成任务，智能体需针对视频与用户请求识别研究问题、检索外部多模态证据，并生成可链接到相关视频片段的可交互报告。团队同时发布 OmniDR-Bench，涵盖中英文 50 个任务、19 个真实子类、6414 条评分标准，并采用 Omni Atomic Rubric Trees 评估视听依据与多模态证据使用。18 种智能体配置中最佳系统得分 63.3/100。
正文 · 原文
Abstract:Deep research agents synthesize open-web evidence into long reports, but current benchmarks do not evaluate research centered on a source video. We introduce Video-Centered Deep Research Report Generation. Given a video and user request, an agent identifies research questions, retrieves external multimodal evidence, and produces an interactive report linked to relevant video moments. This task has direct practical value because it helps users locate relevant moments, verify claims, reproduce demonstrated procedures, and make informed decisions from video content. We present a skill-based workflow and OmniDR-Bench for this task. For evaluation, we propose Omni Atomic Rubric Trees (Omni-ARTs). Each tree contains three dimensions, 23 shared directions, and task-specific atomic binary criteria. The directions test audiovisual grounding, multimodal evidence use, and interactive presentation. Audited Rubric Construction (ARC) builds these trees through independent evidence collection, automated checks, behavioral calibration, and iterative human review. OmniDR-Bench contains 50 Chinese and English tasks across 19 realworld subcategories. It includes 6,414 criteria. Across 18 agent configurations, the best system scores 63.3 out of 100. Current agents still need broader content coverage, stronger causal and logical analysis across modalities, and more accurate citation support. We further analyze models, harnesses, skill configurations, video categories, and research trajectories. The code and data are released at this https URL.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.31158 [cs.CV] |
| (or arXiv:2610.31158v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.31158 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jiahao Meng [view email]
[v1]
Thu, 22 Oct 2026 12:15:30 UTC (9,444 KB)
来源：HuggingFace Daily Papers · arxiv.org