⌁ Awesome World Model Hub Papers · Datasets · Projects
中文 GitHub
← Back to papers

Pre-Trained Video Generative Models as World Simulators

arXiv 25.2 2025 74.8 method

TLDR

Proposes Dynamic World Simulation (DWS) to turn pre-trained video generative models into controllable world simulators using action-conditioned modules and motion-reinforced loss.

Reasoning

Strengths: novel lightweight action-conditioned module, motion-reinforced loss for dynamic consistency, and versatility across diffusion and autoregressive models. Weaknesses: reliance on pre-trained models, limited to action-conditioned scenarios, and potential challenges in complex real-world dynamics.

Read-first score

Read-first score 74.8, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 64.

Topical relevance 42%
91.4

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Recency 8%
86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Methodology quality 25%
80

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=dataset,evaluation,experiment

Reproducibility 25%
38

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=dataset

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 83.

Keyword Scores

world simulator
10
world model
9
generative world model
9
interactive world model
9
video world model
9
world dynamics prediction
9
model-based reinforcement learning world model
9

Deep Analysis

Innovations

  • Dynamic World Simulation (DWS) approach to transform pre-trained video generative models into controllable world simulators
  • Lightweight, universal action-conditioned module that integrates into any existing model for precise action-visual alignment
  • Motion-reinforced loss to enhance action controllability by focusing on dynamic changes
  • Prioritized imagination to improve sample efficiency in model-based reinforcement learning

Methodology

DWS introduces a lightweight action-conditioned module that can be plugged into pre-trained video generative models (both diffusion and autoregressive transformers) to enable action-controlled video generation. A motion-reinforced loss is used to train the module by compelling the model to capture dynamic transitions rather than static visual details. The approach is evaluated on games and robotics domains, and for downstream tasks, prioritized imagination is applied to improve sample efficiency in model-based RL.

Key Results

DWS achieves significant improvements in generating action-controllable, dynamically consistent videos across games and robotics domains. When applied to model-based reinforcement learning, prioritized imagination yields competitive performance compared with state-of-the-art methods.

Limitations

  • Focus on dynamic transition modeling may sacrifice fine-grained visual details
  • Lightweight, universal action-conditioned module may have limited capacity for highly specialized domains
  • Relies on pre-trained video generative models, inheriting their biases and potential failure modes
  • Evaluation is limited to games and robotics; generalizability to other interactive scenarios is not demonstrated

Resources

Paper

Tags