PAVXploreRL: Physical-Action-Visual World Model Reinforcement Learning with Action Exploration

arXiv 2026 57 method

TLDR

PAVXploreRL uses reinforcement learning to optimize physical plausibility, action adherence, and visual fidelity in world models with action exploration.

Reasoning

The paper introduces a novel RL framework that explicitly optimizes three key objectives (PAV) for world models, improving generalization to out-of-distribution actions. Strengths include clear problem formulation and empirical gains over baselines, but weaknesses include lack of real-world validation and reliance on simulated benchmarks.

Read-first score

Read-first score 57, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 54.

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=True; dataset=False; markers=code,github

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,benchmark,evaluation,experiment

Topical relevance 29%

77.1

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Citation impact 18%

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. cited_by_count=0

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FrontierMethodology anchorReproducibility anchor

Rank sensitivity

Stability: volatile; rank range: 849.

Keyword Scores

world model

model-based reinforcement learning world model

video world model

world dynamics prediction

world simulator

interactive world model

generative world model

Deep Analysis

Innovations

Explicit optimization of Physical Plausibility, Action Adherence, and Visual Fidelity (PAV) objectives via reward-driven reinforcement learning on a pretrained latent world model
Noise-driven out-of-distribution action exploration without paired video supervision, jointly leveraging in-distribution trajectories to improve action generalization
Reduction of overestimation bias in policy evaluation compared to expert-only world models like Ctrl-World

Methodology

PAVXploreRL builds on a pretrained latent world model and uses reinforcement learning with reward-driven training to explicitly optimize Physical Plausibility, Action Adherence, and Visual Fidelity. It combines in-distribution expert trajectories with noise-driven out-of-distribution action exploration, without paired video supervision for OOD actions.

Key Results

PAVXploreRL achieves a 5.6% average gain over pretrained baselines across benchmarks, produces higher-quality PAV properties, and yields more reliable policy performance estimates while reducing overestimation bias of Ctrl-World.

Resources

Paper Code