World Models' Last Exam in Physics

arXiv 2026 40.3 benchmark

TLDR

Introduces a measurement-based benchmark of 40 physics tasks evaluating physical consistency of video world models, revealing persistent errors across eight video generation models.

Reasoning

Strengths include broad physics coverage, interpretable quantitative scoring, and validation via synthetic videos and human agreement. Limitations include validity evidence confined to synthetic conditions, leaving real-world physical fidelity less certain.

Read-first score

Read-first score 40.3, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 46.

Recency 5%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Methodology quality 15%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,benchmark,evaluation,experiment

Topical relevance 25%

65.7

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 15%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Graph prestige 15%

5.7

PageRank-style prestige over the local citation graph built from available OpenAlex references. local_pagerank=0.0016

Citation impact 15%

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. cited_by_count=0

Citation velocity 10%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 545.

Keyword Scores

video world model

world model

generative world model

world dynamics prediction

world simulator

model-based reinforcement learning world model

interactive world model

Deep Analysis

Innovations

Introduces World Models' Last Exam in Physics, a measurement-based benchmark for evaluating physical consistency in video world models.
Covers 40 controlled tasks spanning mechanics, optics, fluids, thermal and phase-change phenomena, electromagnetism, and surface tension.
Uses initial image and generation prompt pairs with predefined physical criteria to test observable physical relationships without requiring reference videos.
Combines task-observability screening with task-specific quantitative physical measurements in the evaluator.
Validates the measurement module on synthetic videos with known physical relationships and benchmarks evaluator-human agreement against a direct vision-language model baseline.

Methodology

The benchmark comprises 40 controlled tasks, each pairing an initial image and a generation prompt with predefined physical criteria. Evaluation applies task-observability screening followed by task-specific quantitative physical measurements. The study tests eight video generation models across 1,280 videos, validates the measurement module on synthetic videos with known physical relationships, and compares evaluator-human agreement with a direct vision-language model baseline.

Key Results

Across eight video generation models and 1,280 videos, the best model achieved an overall score of 57.76 out of 100, with persistent physical inconsistencies and substantial variation across tasks. Synthetic-video evaluation supported the measurement module's validity, and the evaluator achieved higher agreement with human judgments than a direct vision-language model baseline in within-task rankings and pairwise comparisons.

Limitations

The abstract notes explicit measurement limitations but does not enumerate them in detail.
The benchmark's coverage is bounded by its 40 tasks across the listed physical domains, and experiments cover eight video generation models.

Resources

Paper