Are Video Models Ready as Zero-Shot Reasoners? An Empirical Study with the MME-CoF Benchmark

arXiv 2025 57.2 benchmark

TLDR

This paper introduces MME-CoF, a benchmark evaluating Veo-3's zero-shot visual reasoning across 12 dimensions, finding video models are not yet reliable standalone reasoners.

评分理由

The study provides a systematic empirical evaluation across multiple reasoning dimensions and introduces a reusable benchmark, which are clear strengths. However, it focuses on a single video model and relies only on the described benchmark without broader comparisons, limiting generalizability.

Read-first 评分解释

综合优先阅读分 57.2，由主题、引用、图谱、方法、可复现性和近期性等信号加权得到。原始总分保留为 31。

近期性 8%

86.7

使用温和的时间衰减，让近期论文更容易浮现，同时保留较早基础工作的价值。年份：2025

方法质量 25%

检查可见的摘要与分析字段，寻找实验、数据集、基线、指标和局限性等方法证据。命中信号：基准、评估、指标

可复现性 25%

检查链接和可见文本中的论文、代码、数据集、工件与仓库信号。论文：有；代码：无；数据：无；命中信号：代码、GitHub

主题相关性 42%

44.3

使用现有 LLM 关键词相关性评分，并归一化到 0-100。关键词：world model、world simulator、generative world model、interactive world model、video world model、world dynamics prediction、model-based reinforcement learning world model

研究版图角色

前沿论文方法锚点

排序敏感性

稳定性：volatile；排名波动范围：379。

关键词评分

video world model

world model

world dynamics prediction

generative world model

world simulator

interactive world model

model-based reinforcement learning world model

深度分析

创新点

Introduction of MME-CoF, a compact benchmark designed for in-depth assessment of Chain-of-Frame reasoning in video models.
A comprehensive empirical study evaluating Veo-3 as a zero-shot reasoner across 12 reasoning dimensions including spatial, geometric, physical, temporal, and embodied logic.
Systematic characterization of both strengths and failure modes of a leading video generation model in visual reasoning scenarios.

方法

The authors evaluate the video model Veo-3 as a zero-shot reasoner across 12 dimensions covering spatial, geometric, physical, temporal, and embodied logic. They curate evaluation data into MME-CoF, a compact benchmark for Chain-of-Frame reasoning assessment. The study systematically characterizes reasoning behavior, strengths, and failure modes.

关键结果

Veo-3 shows promising reasoning patterns on short-horizon spatial coherence, fine-grained grounding, and locally consistent dynamics, but remains limited in long-horizon causal reasoning, strict geometric constraints, and abstract logic. Overall, it is not yet reliable as a standalone zero-shot reasoner but shows encouraging signs as a complementary visual engine.

局限性

The study focuses on a single model, Veo-3, so findings may not generalize to other video models.
Current video models remain limited in long-horizon causal reasoning, strict geometric constraints, and abstract logic.
Video models are not yet reliable as standalone zero-shot reasoners.
MME-CoF is a compact benchmark, which may constrain the breadth of evaluation.

资源

论文