SmallWorlds: Assessing Dynamics Understanding of World Models in Isolated Environments

arXiv 25.11 2025 59.3 benchmark

TLDR

Introduces SmallWorld Benchmark for systematically evaluating world models' dynamics understanding in isolated, controlled environments.

Reasoning

Strengths include a novel benchmark for controlled evaluation and testing across multiple architectures. Weaknesses are the limitation to fully observable state spaces and lack of real-world validation.

Read-first score

Read-first score 59.3, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 37.

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark,evaluation,experiment,result

Recency 8%

86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Topical relevance 42%

52.9

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 279.

Keyword Scores

world model

world dynamics prediction

generative world model

model-based reinforcement learning world model

world simulator

interactive world model

video world model

Deep Analysis

Innovations

Introduction of the SmallWorld Benchmark, a testbed for assessing world model capability under isolated and precisely controlled dynamics without handcrafted reward signals.
Comprehensive evaluation of four representative architectures (Recurrent State Space Model, Transformer, Diffusion model, Neural ODE) across six distinct domains in a fully observable state space.

Methodology

The authors design the SmallWorld Benchmark as a controlled testbed with isolated dynamics, then conduct experiments in a fully observable state space on four model architectures: Recurrent State Space Model, Transformer, Diffusion model, and Neural ODE. They evaluate how well these models capture environment structure and how predictions degrade over extended rollouts across six domains.

Key Results

The experiments reveal how effectively each architecture captures environment structure and show that predictions deteriorate over extended rollouts, highlighting both strengths and limitations of current modeling paradigms.

Limitations

Evaluation is limited to fully observable state space, not addressing partially observable or real-world environments.
Only six distinct domains are tested, which may not cover the full range of possible dynamics.
Only four representative architectures are examined, leaving out other potential world model designs.
The benchmark does not incorporate handcrafted reward signals, which may limit comparability to reinforcement learning settings where rewards are used.

Resources

Paper