World Modeling Makes a Better Planner: Dual Preference Optimization for Embodied Task Planning

arXiv 25.3 2025 48.2 method, application

TLDR

Proposes Dual Preference Optimization for embodied task planning, jointly optimizing state prediction and action selection via preference learning, outperforming baselines on VoTa-Bench.

Reasoning

Strengths include a novel framework combining world modeling with preference learning and automatic data collection via tree search, with strong empirical results. Weaknesses are limited evaluation to a single benchmark and lack of discussion on generalization or limitations.

Read-first score

Read-first score 48.2, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 31.

Recency 8%

86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark,experiment

Topical relevance 42%

44.3

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Field roles

Frontier

Rank sensitivity

Stability: volatile; rank range: 406.

Keyword Scores

world model

world dynamics prediction

model-based reinforcement learning world model

world simulator

generative world model

interactive world model

video world model

Deep Analysis

Innovations

Dual Preference Optimization (D^2PO) framework that jointly optimizes state prediction and action selection through preference learning
Automatic collection of trajectories and stepwise preference data using a tree search mechanism without human annotation
Demonstrated improvement over existing methods and GPT-4o when applied to multiple LVLMs (Qwen2-VL, LLaVA, LLaMA) on VoTa-Bench

Methodology

Dual Preference Optimization (D^2PO) is a learning framework that jointly optimizes state prediction and action selection via preference learning. It employs a tree search mechanism to automatically generate trajectories and stepwise preference data for training, eliminating the need for human annotation. The method is applied to LVLMs (Qwen2-VL 7B, LLaVA-1.6 7B, LLaMA-3.2 11B) and evaluated on the VoTa-Bench benchmark.

Key Results

On VoTa-Bench, D^2PO-based methods significantly outperform existing methods and GPT-4o across three LVLM backbones, achieving superior task success rates with more efficient execution paths.

Resources

Paper