WorldSample: Closed-loop Real-robot RL with World Modelling

arXiv 2026 44 method, application

TLDR

WorldSample improves real-robot RL by using a world model for synthetic data augmentation and Policy-Paced Learning, boosting success rate by 28% and reducing training steps by 59%.

Reasoning

The paper presents a novel closed-loop framework that integrates real rollouts with a post-trained world model to generate high-fidelity synthetic transitions, addressing high interaction costs. Strengths include significant empirical gains on manipulation tasks and a principled method (PPL) to mitigate hallucination noise. Weaknesses are limited task scope and potential sensitivity to world model quality, though the abstract does not discuss failure cases.

Read-first score

Read-first score 44, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 53.

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Topical relevance 29%

75.7

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,experiment

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Citation impact 18%

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. cited_by_count=0

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

Frontier

Rank sensitivity

Stability: volatile; rank range: 431.

Keyword Scores

world model

generative world model

world dynamics prediction

model-based reinforcement learning world model

world simulator

interactive world model

video world model

Deep Analysis

Innovations

Closed-loop real-synthetic loop between physical rollouts, world-model generation, and policy improvement for real-robot RL
Policy-Paced Learning (PPL) to regulate training via sample selection and scheduling, balancing augmentation against value overestimation and hallucination noise
Post-trained world model grounded on real rollouts to generate high-fidelity synthetic transitions, reducing visual hallucination

Methodology

WorldSample is a data augmentation framework that uses a world model post-trained on real robot rollouts to generate synthetic transitions, forming a real-synthetic loop. It introduces Policy-Paced Learning to select and schedule these synthetic samples during RL training, mitigating value overestimation and hallucination noise. Evaluated on real-robot contact-rich and precise manipulation tasks against baselines.

Key Results

WorldSample improves policy success rate by 28% and reduces training steps by 59% compared to baselines, while improving world model visual fidelity by 19.4dB PSNR and 0.47 SSIM over demonstration-only post-training.

Resources

Paper