Offline Robotic World Model: Learning Robotic Policies without a Physics Simulator

arXiv 25.4 2025 64.1 method

TLDR

Offline MBRL with uncertainty-aware autoregressive world models enables effective policy learning on real robots without simulators.

Reasoning

The paper introduces RWM-U, an autoregressive world model with epistemic uncertainty estimation, combined with MOPO-PPO for offline MBRL. Strengths include real-world validation on quadruped and humanoid robots and consistent outperformance of baselines. Weaknesses are limited discussion of failure modes and comparison to other world model architectures.

Read-first score

Read-first score 64.1, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 46.

Recency 8%

86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,benchmark,dataset,result

Topical relevance 42%

65.7

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=dataset

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 109.

Keyword Scores

world model

model-based reinforcement learning world model

world dynamics prediction

generative world model

interactive world model

world simulator

video world model

Deep Analysis

Innovations

RWM-U: an autoregressive world model with epistemic uncertainty estimation for temporally consistent multi-step rollouts with uncertainty propagation over long horizons
MOPO-PPO: adaptation of uncertainty-penalized policy optimization to the stable, on-policy PPO framework for real-world control
A principled pipeline for making offline model-based reinforcement learning effective on physical robots without requiring a physics simulator

Methodology

The paper proposes RWM-U, an autoregressive world model that incorporates epistemic uncertainty estimation to enable temporally consistent multi-step rollouts with propagated uncertainty. This is combined with MOPO-PPO, which adapts uncertainty-penalized policy optimization to the on-policy PPO framework. The approach is evaluated on diverse manipulation and locomotion tasks in simulation and on real quadruped and humanoid robots, training policies entirely from offline datasets.

Key Results

The proposed method consistently outperforms model-free and uncertainty-unaware model-based baselines. Fusing real-world data in model learning yields robust policies that surpass online model-free baselines trained solely in simulation.

Resources

Paper