ManipDreamer: Boosting Robotic Manipulation World Model with Action Tree and Visual Guidance

arXiv 25.4 2025 57.8 method

TLDR

ManipDreamer improves robotic manipulation world models using action tree for instruction-following and visual guidance for video quality.

Reasoning

The paper introduces a novel action tree representation to capture relationships between instruction primitives and a visual guidance adapter combining depth and semantic cues, addressing key limitations in prior work. Strengths include clear methodology and benchmark evaluations; weaknesses are lack of explicit discussion on limitations or real-world deployment challenges.

Read-first score

Read-first score 57.8, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 43.

Recency 8%

86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark,evaluation,metric

Topical relevance 42%

61.4

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 269.

Keyword Scores

world model

video world model

generative world model

world dynamics prediction

interactive world model

world simulator

model-based reinforcement learning world model

Deep Analysis

Innovations

Representing instruction as an action tree to capture relationships between instruction primitives, with embeddings assigned to tree nodes for guiding the world model.
Introducing a visual guidance adapter that combines depth and semantic guidance to enhance temporal and physical consistency in video generation.

Methodology

ManipDreamer builds a world model conditioned on an action tree representation of instructions, where each node embedding is obtained by navigating the tree. A visual guidance adapter integrates depth and semantic cues to improve video quality. The model is evaluated on robotic manipulation benchmarks against RoboDreamer using metrics like PSNR, SSIM, Flow Error, and task success rate.

Key Results

On unseen tasks, ManipDreamer improves PSNR from 19.55 to 21.05, SSIM from 0.7474 to 0.7982, and reduces Flow Error from 3.506 to 3.201 compared to RoboDreamer. It also increases the average success rate by 2.5% across 6 RLbench tasks.

Resources

Paper