⌁ Awesome World Model Hub Papers · Datasets · Projects
中文 GitHub
← Back to papers

Are Video Models Ready as Zero-Shot Reasoners? An Empirical Study with the MME-CoF Benchmark

arXiv 2025 57.2 benchmark

TLDR

This paper introduces MME-CoF, a benchmark evaluating Veo-3's zero-shot visual reasoning across 12 dimensions, finding video models are not yet reliable standalone reasoners.

Reasoning

The study provides a systematic empirical evaluation across multiple reasoning dimensions and introduces a reusable benchmark, which are clear strengths. However, it focuses on a single video model and relies only on the described benchmark without broader comparisons, limiting generalizability.

Read-first score

Read-first score 57.2, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 31.

Recency 8%
86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Methodology quality 25%
80

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark,evaluation,metric

Reproducibility 25%
46

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=code,github

Topical relevance 42%
44.3

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 379.

Keyword Scores

video world model
7
world model
6
world dynamics prediction
6
generative world model
5
world simulator
4
interactive world model
2
model-based reinforcement learning world model
1

Deep Analysis

Innovations

  • Introduction of MME-CoF, a compact benchmark designed for in-depth assessment of Chain-of-Frame reasoning in video models.
  • A comprehensive empirical study evaluating Veo-3 as a zero-shot reasoner across 12 reasoning dimensions including spatial, geometric, physical, temporal, and embodied logic.
  • Systematic characterization of both strengths and failure modes of a leading video generation model in visual reasoning scenarios.

Methodology

The authors evaluate the video model Veo-3 as a zero-shot reasoner across 12 dimensions covering spatial, geometric, physical, temporal, and embodied logic. They curate evaluation data into MME-CoF, a compact benchmark for Chain-of-Frame reasoning assessment. The study systematically characterizes reasoning behavior, strengths, and failure modes.

Key Results

Veo-3 shows promising reasoning patterns on short-horizon spatial coherence, fine-grained grounding, and locally consistent dynamics, but remains limited in long-horizon causal reasoning, strict geometric constraints, and abstract logic. Overall, it is not yet reliable as a standalone zero-shot reasoner but shows encouraging signs as a complementary visual engine.

Limitations

  • The study focuses on a single model, Veo-3, so findings may not generalize to other video models.
  • Current video models remain limited in long-horizon causal reasoning, strict geometric constraints, and abstract logic.
  • Video models are not yet reliable as standalone zero-shot reasoners.
  • MME-CoF is a compact benchmark, which may constrain the breadth of evaluation.

Resources

Paper

Tags