YoCausal: How Far is Video Generation from World Model? A Causality Perspective
TLDR
YoCausal benchmarks video diffusion models' causal understanding using reversed real-world videos, revealing a gap between temporal perception and true causality.
Reasoning
The paper introduces a novel benchmark (YoCausal) with two levels to disentangle temporal bias from causal reasoning, using real-world videos and cognitive science principles. Strengths include a zero-cost counterfactual generation method and evaluation of 13 models; weaknesses are that the benchmark's scalability and human-level comparison details are not fully elaborated in the abstract.
Read-first score
Read-first score 57.3, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 40.
Field roles
Rank sensitivity
Stability: volatile; rank range: 352.
Keyword Scores
Deep Analysis
Innovations
- Two-level benchmark inspired by the Violation of Expectation (VoE) paradigm from cognitive science
- Reverse Surprise Index (RSI) for quantifying arrow-of-time perception via denoising loss
- Causality Cognition Index (CCI) using a VLM to stratify datasets into causal and non-causal subsets
- Zero-cost generation of natural counterfactual samples by temporally reversing real-world videos
Methodology
YoCausal is a two-level benchmark that evaluates video diffusion models on causal understanding. Level 1 uses the Reverse Surprise Index (RSI), which measures arrow-of-time perception by comparing denoising losses on original and temporally reversed videos. Level 2 introduces the Causality Cognition Index (CCI), which leverages a vision-language model (VLM) to partition datasets into causal and non-causal subsets, enabling disentanglement of genuine causal reasoning from temporal bias. The benchmark is arbitrarily extensible as it uses real-world videos reversed at zero cost.
Key Results
Evaluation of 13 state-of-the-art video diffusion models reveals that perceiving the arrow of time does not imply understanding causality, and a significant gap persists relative to human-level causal cognition.