FlowDreamer: A RGB-D World Model with Flow-based Motion Representations for Robot Manipulation

arXiv 25.5 2025 71.3 method

TLDR

FlowDreamer uses 3D scene flow as explicit motion representations in an RGB-D world model for robot manipulation, outperforming baselines on video prediction and planning tasks.

Reasoning

The paper introduces a novel explicit motion representation (3D scene flow) within a modularized yet end-to-end trained world model, showing clear improvements across multiple benchmarks. However, it lacks explicit mention of real-world experiments, and the scope is limited to RGB-D inputs and robot manipulation domains.

Read-first score

Read-first score 71.3, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 54.

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,benchmark,evaluation,experiment,result

Recency 8%

86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Topical relevance 42%

77.1

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=github

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 53.

Keyword Scores

world model

video world model

world dynamics prediction

generative world model

interactive world model

model-based reinforcement learning world model

world simulator

Deep Analysis

Innovations

Introduces explicit 3D scene flow as motion representation for RGB-D world models, replacing implicit dynamics prediction.
Modularized architecture with a U-Net for flow prediction and a diffusion model for future frame generation, trained end-to-end.
Demonstrates consistent improvements across multiple benchmarks in video prediction and visual planning tasks.

Methodology

FlowDreamer first uses a U-Net to predict 3D scene flow from past RGB-D frames and robot actions. Then a diffusion model generates future frames conditioned on the predicted scene flow. The entire model is trained end-to-end despite its modular design. Evaluation is conducted on four benchmarks covering video prediction and visual planning tasks.

Key Results

FlowDreamer outperforms baseline RGB-D world models by 7% on semantic similarity, 11% on pixel quality, and 6% on success rate across various robot manipulation domains.

Resources

Paper Project