⌁ Awesome Auto Research Hub Papers · Datasets · Projects
中文 GitHub
← Back to papers

Beyond Watching and Searching: Benchmarking Agents for Video-centered Deep Research Reports

arXiv 2026 53 method

TLDR

Introduces OmniDR-Bench and Omni-ARTs for benchmarking agents that generate interactive deep research reports centered on source videos, using 50 multilingual real-world tasks.

Reasoning

The paper addresses a novel benchmark gap with a carefully designed evaluation framework and real-world tasks. Its main limitation is that current agents still achieve modest performance, indicating substantial room for improvement.

Read-first score

Read-first score 53, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 35.

Recency 8%
100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Methodology quality 25%
80

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=analysis,benchmark,evaluation

Reproducibility 25%
50

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=code,github,reproduce

Topical relevance 42%
29.2

Uses existing LLM keyword relevance scores normalized to 0-100. AI scientist,automated scientific discovery,autonomous research agent,automated research,literature review agent,survey generation,automated experimentation,experiment design agent,AI for scientific research,paper writing agent,research automation,scientific discovery agent

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 139.

Keyword Scores

autonomous research agent
8
automated research
8
research automation
8
survey generation
3
literature review agent
2
AI for scientific research
2
AI scientist
1
automated scientific discovery
1
paper writing agent
1
scientific discovery agent
1
automated experimentation
0
experiment design agent
0

Deep Analysis

Innovations

  • Introduces Video-Centered Deep Research Report Generation, where agents produce interactive reports linked to relevant video moments.
  • Presents OmniDR-Bench, a benchmark with 50 Chinese and English tasks across 19 real-world subcategories and 6,414 criteria.
  • Proposes Omni Atomic Rubric Trees (Omni-ARTs), with three dimensions, 23 shared directions, and task-specific atomic binary criteria.
  • Proposes Audited Rubric Construction (ARC), which builds rubric trees through independent evidence collection, automated checks, behavioral calibration, and iterative human review.
  • Describes a skill-based workflow for video-centered deep research agents.

Methodology

The benchmark defines a task in which, given a video and user request, an agent identifies research questions, retrieves external multimodal evidence, and produces an interactive report linked to relevant video moments. Evaluation uses Omni-ARTs rubric trees constructed via ARC, and the benchmark evaluates 18 agent configurations across 50 bilingual tasks in 19 real-world subcategories.

Key Results

Across 18 agent configurations, the best system scores 63.3 out of 100. Current agents still need broader content coverage, stronger causal and logical analysis across modalities, and more accurate citation support.

Limitations

  • Current agents still need broader content coverage.
  • Stronger causal and logical analysis across modalities is needed.
  • More accurate citation support is needed.

Resources

Paper

Tags