Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation

arXiv 26.3 2026 72.7 method, application

TLDR

WorldDrive unifies vision and motion representation in a driving world model to couple scene generation and real-time planning.

Reasoning

Strengths include a novel trajectory-conditioned world model that bridges scene generation and planning, with a future-aware rewarder. Weaknesses: the abstract lacks explicit real-world validation or benchmark results, making empirical support unclear.

Read-first score

Read-first score 72.7, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 49.

Recency 8%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=True; dataset=False; markers=code,github

Topical relevance 42%

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark,experiment

Field roles

FrontierReproducibility anchor

Rank sensitivity

Stability: volatile; rank range: 90.

Keyword Scores

world model

world dynamics prediction

generative world model

model-based reinforcement learning world model

video world model

interactive world model

world simulator

Deep Analysis

Innovations

Trajectory-aware Driving World Model that conditions on a trajectory vocabulary to enforce consistency between visual dynamics and motion intentions, enabling diverse and plausible future scene generation conditioned on a specific trajectory.
Transfer of vision and motion encoders from the world model to a downstream Multi-modal Planner, ensuring the driving policy operates on mature representations pre-optimized by scene generation.
Future-aware Rewarder that distills future latent representation from the frozen world model to evaluate and select optimal trajectories in real-time.

Methodology

WorldDrive is a holistic framework that couples scene generation and real-time planning by unifying vision and motion representation. It introduces a Trajectory-aware Driving World Model that uses a trajectory vocabulary to enforce consistency between visual dynamics and motion intentions. The vision and motion encoders are transferred to a Multi-modal Planner, and a Future-aware Rewarder leverages the frozen world model's foresight to evaluate and select optimal trajectories. The framework is evaluated on NAVSIM, NAVSIM-v2, and nuScenes benchmarks.

Key Results

WorldDrive achieves leading planning performance among vision-only methods while maintaining high-fidelity action-controlled video generation capabilities, demonstrating the effectiveness of unifying vision and motion representation for robust autonomous driving.

Resources

Paper Code