⌁ Awesome World Model Hub Papers · Datasets · Projects
中文 GitHub
← Back to papers

VaViM and VaVAM: Autonomous Driving through Video Generative Modeling

arXiv 25.2 2025 63.5 method, system, application

TLDR

Introduces open-source auto-regressive video models VaViM and VaVAM for autonomous driving, showing video pre-training transfers to real-world driving.

Reasoning

The paper presents a clear, simple architecture and provides open-source code, which are strengths. However, it focuses narrowly on driving and does not claim a general world model, limiting the relevance of some keywords. The evaluation includes real-world driving scenarios, supporting 'has_real_world'.

Read-first score

Read-first score 63.5, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 21.

Methodology quality 25%
90

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=analysis,dataset,evaluation,metric

Recency 8%
86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Reproducibility 25%
85

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=True; dataset=False; markers=code,dataset,github

Topical relevance 42%
30

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Field roles

FrontierMethodology anchorReproducibility anchor

Rank sensitivity

Stability: volatile; rank range: 720.

Keyword Scores

video world model
6
world dynamics prediction
5
world model
3
world simulator
2
generative world model
2
model-based reinforcement learning world model
2
interactive world model
1

Deep Analysis

Innovations

  • Open-source auto-regressive video model (VaViM) for autonomous driving that predicts frames using spatio-temporal token sequences
  • Video-action model (VaVAM) that leverages VaViM's learned representations to generate driving trajectories via imitation learning
  • Complete perception-to-action pipeline from video generation to driving actions
  • Investigation of how video pre-training transfers to real-world driving in open- and closed-loop scenarios

Methodology

VaViM is an auto-regressive video model that predicts future frames by modeling spatio-temporal token sequences. VaVAM uses the representations learned by VaViM to perform imitation learning for generating driving trajectories. The models are evaluated in both open-loop and closed-loop driving scenarios, with analysis of scaling effects and safety metrics.

Key Results

Video-based pre-training shows promise for autonomous driving, with learned representations exhibiting semantic richness. Scaling benefits for video synthesis are observed, but the relationship between model size, data, and safety metrics in closed-loop evaluations is complex and not fully resolved.

Limitations

  • The relationship between model size, data, and safety metrics in closed-loop evaluations is complex and not fully understood
  • Evaluation is limited to driving scenarios; generalization to other autonomous tasks is not demonstrated
  • The abstract does not specify the dataset or evaluation environment, limiting reproducibility assessment
  • The video-action model relies on imitation learning, which may not handle all edge cases in real-world driving

Resources

PaperCode

Tags