⌁ Awesome World Model Hub Papers · Datasets · Projects
中文 GitHub
← Back to papers

Raw2Drive: Reinforcement Learning with Aligned World Models for End-to-End Autonomous Driving (in CARLA v2)

arXiv 25.5 2025 61.3 method, application

TLDR

Raw2Drive uses dual-stream MBRL with aligned world models to achieve state-of-the-art end-to-end autonomous driving in CARLA v2.

Reasoning

The paper introduces a novel dual-stream MBRL approach that aligns a privileged world model with a raw sensor world model via a guidance mechanism, addressing the gap of using raw sensor data in MBRL for autonomous driving. Its strength lies in achieving SOTA on CARLA Leaderboard 2.0 and Bench2Drive, but it is limited to simulation environments with no real-world validation.

Read-first score

Read-first score 61.3, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 53.

Recency 8%
86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Topical relevance 42%
75.7

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Methodology quality 25%
60

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=result

Reproducibility 25%
30

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Field roles

Frontier

Rank sensitivity

Stability: volatile; rank range: 375.

Keyword Scores

world model
10
model-based reinforcement learning world model
10
world dynamics prediction
9
world simulator
7
generative world model
6
video world model
6
interactive world model
5

Deep Analysis

Innovations

  • Dual-stream MBRL approach combining a privileged world model and a raw sensor world model for end-to-end autonomous driving
  • Guidance Mechanism to ensure consistency between raw sensor and privileged world models during rollouts
  • Using prior knowledge from privileged world model heads to guide raw sensor policy training
  • First RL-based end-to-end method on CARLA Leaderboard 2.0 and Bench2Drive, achieving state-of-the-art performance

Methodology

Raw2Drive first trains an auxiliary privileged world model paired with a neural planner using privileged information as input. Then a raw sensor world model is trained via a Guidance Mechanism that enforces consistency between the raw sensor and privileged world models during rollouts. Finally, the raw sensor world model leverages prior knowledge from the heads of the privileged world model to guide the training of the raw sensor policy.

Key Results

Raw2Drive is the only RL-based end-to-end method on CARLA Leaderboard 2.0 and Bench2Drive, and it achieves state-of-the-art performance.

Limitations

  • Requires privileged information for initial training of the auxiliary world model, which may not be available in real-world settings
  • Evaluated only in simulation (CARLA), so real-world applicability and generalization are not demonstrated
  • Potential complexity and computational overhead of the dual-stream training and Guidance Mechanism

Resources

Paper

Tags