DiWA: Diffusion Policy Adaptation with World Models

CoRL 25 2025 57.2 method, application

TLDR

Fine-tuning diffusion policies with reinforcement learning (RL) presents significant challenges.

Reasoning

Fallback reasoning generated from available title and abstract metadata: Fine-tuning diffusion policies with reinforcement learning (RL) presents significant challenges. The long denoising sequence for each action prediction impedes effective reward propagation. Moreover, standard RL methods require millions of real-world interactions, posing a major bottleneck for practical fine-tuning....

Read-first score

Read-first score 57.2, weighted from topical fit, citation, graph, method, reproducibility, and recency signals.

Methodology quality 25%

100

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,benchmark,dataset,evaluation,result

Recency 8%

86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=code,dataset

Topical relevance 42%

32.4

Matches configured research keywords against title, abstract, tags, and analysis text. matched=7

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 599.

Deep Analysis

Innovations

Introduces a world model to enable offline fine-tuning of diffusion policies with reinforcement learning, eliminating the need for millions of real-world interactions.
Achieves dramatically improved sample efficiency by training the world model once on a few hundred thousand offline play interactions.
First demonstration of fine-tuning diffusion policies for real-world robotic skills using an offline world model.

Methodology

DiWA leverages a world model trained once on a few hundred thousand offline play interactions. It then uses reinforcement learning to fine-tune diffusion-based robotic policies entirely offline, avoiding the need for millions of environment interactions that model-free approaches require.

Key Results

On the CALVIN benchmark, DiWA improves performance across eight tasks using only offline adaptation, while requiring orders of magnitude fewer physical interactions than model-free baselines.

Limitations

Requires a pre-collected offline dataset of a few hundred thousand play interactions for world model training.
Evaluation is limited to the CALVIN benchmark; generalization to other tasks or real-world scenarios is not demonstrated.
Offline adaptation may struggle with large distribution shifts not covered by the offline data.

Resources

Paper Project