World Models' Last Exam in Physics

arXiv 2026 40.3 benchmark

TLDR

Introduces a measurement-based benchmark of 40 physics tasks evaluating physical consistency of video world models, revealing persistent errors across eight video generation models.

评分理由

Strengths include broad physics coverage, interpretable quantitative scoring, and validation via synthetic videos and human agreement. Limitations include validity evidence confined to synthetic conditions, leaving real-world physical fidelity less certain.

Read-first 评分解释

综合优先阅读分 40.3，由主题、引用、图谱、方法、可复现性和近期性等信号加权得到。原始总分保留为 46。

近期性 5%

100

使用温和的时间衰减，让近期论文更容易浮现，同时保留较早基础工作的价值。年份：2026

方法质量 15%

检查可见的摘要与分析字段，寻找实验、数据集、基线、指标和局限性等方法证据。命中信号：基线、基准、评估、实验

主题相关性 25%

65.7

使用现有 LLM 关键词相关性评分，并归一化到 0-100。关键词：world model、world simulator、generative world model、interactive world model、video world model、world dynamics prediction、model-based reinforcement learning world model

可复现性 15%

检查链接和可见文本中的论文、代码、数据集、工件与仓库信号。论文：有；代码：无；数据：无；命中信号：无

图谱声望 15%

5.7

基于可用 OpenAlex 引用关系构建局部引用图，并计算 PageRank 风格的图谱声望。局部 PageRank：0.0016

引用影响力 15%

使用 OpenAlex 形态的引用元数据作为文献关注度信号，并与论文本身质量分开处理。引用数：0

引用速度 10%

引用速度按发表年限估算年均引用，降低旧论文天然占优的偏差。年均引用：0.00

研究版图角色

前沿论文方法锚点

排序敏感性

稳定性：volatile；排名波动范围：545。

关键词评分

video world model

world model

generative world model

world dynamics prediction

world simulator

model-based reinforcement learning world model

interactive world model

深度分析

创新点

Introduces World Models' Last Exam in Physics, a measurement-based benchmark for evaluating physical consistency in video world models.
Covers 40 controlled tasks spanning mechanics, optics, fluids, thermal and phase-change phenomena, electromagnetism, and surface tension.
Uses initial image and generation prompt pairs with predefined physical criteria to test observable physical relationships without requiring reference videos.
Combines task-observability screening with task-specific quantitative physical measurements in the evaluator.
Validates the measurement module on synthetic videos with known physical relationships and benchmarks evaluator-human agreement against a direct vision-language model baseline.

方法

The benchmark comprises 40 controlled tasks, each pairing an initial image and a generation prompt with predefined physical criteria. Evaluation applies task-observability screening followed by task-specific quantitative physical measurements. The study tests eight video generation models across 1,280 videos, validates the measurement module on synthetic videos with known physical relationships, and compares evaluator-human agreement with a direct vision-language model baseline.

关键结果

Across eight video generation models and 1,280 videos, the best model achieved an overall score of 57.76 out of 100, with persistent physical inconsistencies and substantial variation across tasks. Synthetic-video evaluation supported the measurement module's validity, and the evaluator achieved higher agreement with human judgments than a direct vision-language model baseline in within-task rankings and pairwise comparisons.

局限性

The abstract notes explicit measurement limitations but does not enumerate them in detail.
The benchmark's coverage is bounded by its 40 tasks across the listed physical domains, and experiments cover eight video generation models.

资源

论文