Training Object Permanence in World Models
TLDR
Introduces WROP, a 1.5M-sample synthetic dataset and exam to train and evaluate object permanence in video generation world models; PWM-WROP ranks top among continuation models.
Reasoning
The paper presents a large, carefully designed synthetic benchmark and evaluates many video models, with strong resource release and empirical rigor. However, the reliance on Blender-generated synthetic tasks may limit generalization to real-world physical scenes, and the abstract does not demonstrate interactive or RL-based world model capabilities.
Read-first score
Read-first score 33.4, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 36.
Field roles
Rank sensitivity
Stability: volatile; rank range: 258.
Keyword Scores
Deep Analysis
Innovations
- WROP data infrastructure with 150 hand-designed cognitive science inspired tasks divided into six cognitive categories for object permanence and solidity.
- Blender-based generators that randomize speed, lighting, camera angle, and other nuisance parameters while preserving each task's cognitive structure, yielding 10,000+ samples per task.
- Release of a 1.5M-sample training corpus and a 300-question exam for evaluating object permanence in video models.
- PWM-WROP, a 16B world model, and PWM, a native-PyTorch training stack on AWS Trainium2.
- Blind pairwise Elo evaluation across 14 video models spanning reference-to-video, edit, and continuation tasks.
Methodology
The authors designed 150 cognitive-science-inspired tasks across six categories and built Blender generators that randomize nuisance parameters such as speed, lighting, and camera angle while preserving cognitive structure. This produced a 1.5M-sample training corpus and a 300-question exam. They evaluated 14 video models, including their 16B PWM-WROP, using a blind pairwise Elo study.
Key Results
On the 300-question WROP exam, PWM-WROP ranks first among continuation models and third overall, behind only a statistical tie between two reference-to-video models.