World-in-World: World Models in a Closed-Loop World

arXiv 25.10 2025 56.6 benchmark, system

TLDR

Introduces World-in-World, a platform for benchmarking world models in closed-loop embodied environments, revealing that controllability and post-training scaling matter more than visual quality.

Reasoning

The paper addresses a clear gap in world model evaluation by introducing a closed-loop benchmark focused on task success rather than visual quality. Its strengths include empirical surprises and a data scaling law, but it is limited to four simulated environments without real-world validation.

Read-first score

Read-first score 56.6, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 41.

Recency 8%

86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark,evaluation,metric

Topical relevance 42%

58.6

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 277.

Keyword Scores

world model

generative world model

world simulator

interactive world model

video world model

model-based reinforcement learning world model

world dynamics prediction

Deep Analysis

Innovations

First open platform benchmarking world models in a closed-loop world that mirrors real agent-environment interactions
Unified online planning strategy and standardized action API enabling heterogeneous world models for decision making
Four closed-loop environments for rigorous evaluation with task success as the primary metric
First data scaling law for world models in embodied settings
Three surprises: visual quality does not guarantee task success (controllability matters more), scaling post-training with action-observation data is more effective than upgrading pretrained video generators, and allocating more inference-time compute substantially improves closed-loop performance

Methodology

World-in-World provides a unified online planning strategy and a standardized action API, enabling heterogeneous world models for decision making. It curates four closed-loop environments that rigorously evaluate diverse world models, prioritizing task success as the primary metric and moving beyond visual quality. The study also presents the first data scaling law for world models in embodied settings.

Key Results

The study uncovers three surprises: (1) visual quality alone does not guarantee task success—controllability matters more; (2) scaling post-training with action-observation data is more effective than upgrading pretrained video generators; and (3) allocating more inference-time compute allows world models to substantially improve closed-loop performance.

Resources

Paper