DriveCtrl: Conditioned Sim-to-Real Driving Video Generation

arXiv 26.5 2026 40.5 method

TLDR

DriveCtrl is a depth-conditioned sim-to-real video generation framework that produces realistic driving videos while preserving annotations.

Reasoning

The paper addresses the domain gap in sim-to-real driving video generation with a structure-aware adapter and scalable pipeline. However, it lacks novelty in world model concepts and the evaluation metric is not fully detailed.

Read-first score

Read-first score 40.5, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 0.

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=dataset,evaluation,experiment,metric

Citation impact 18%

77.9

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. citation_normalized_percentile=0.77937277

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=dataset

Topical relevance 29%

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FoundationFrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 236.

Keyword Scores

world model

world simulator

generative world model

interactive world model

video world model

world dynamics prediction

model-based reinforcement learning world model

Deep Analysis

Innovations

Depth-conditioned controllable sim-to-real driving video generation framework (DriveCtrl) built upon a pretrained video foundation model
Structure-aware adapter that enables depth-guided generation while preserving scene layout and motion patterns of source simulation
Scalable data generation pipeline with three conditioning signals: structural depth, reference-dataset style, and text prompts, while preserving frame-level annotations
Driving Video Realism Score (DVRS), a driving-domain-specific knowledge-informed evaluation metric for assessing realism of generated videos

Methodology

DriveCtrl is built upon a pretrained video foundation model and introduces a structure-aware adapter for depth-guided generation. It uses a scalable data generation pipeline that transforms simulator videos into realistic driving footage matching the visual style of a target real-world dataset, supporting conditioning on structural depth, reference-dataset style, and text prompts while preserving frame-level annotations. The framework is evaluated using the proposed DVRS metric alongside standard realism, temporal quality, and perception task performance metrics.

Key Results

DriveCtrl consistently outperforms the base model and competing alternatives in realism, temporal quality, and perception task performance, substantially narrowing the sim-to-real gap for driving video generation.

Resources

Paper