Think Before You Drive: World Model-Inspired Multimodal Grounding for Autonomous Vehicles

arXiv 25.12 2025 39.1 method, application

TLDR

A world model-inspired framework for autonomous vehicle visual grounding that reasons about future spatial states to disambiguate natural-language commands.

Reasoning

The paper introduces a novel integration of world model principles for spatial reasoning in autonomous driving, supported by strong empirical results across multiple benchmarks and a new dataset. However, the abstract lacks discussion of limitations and the world model is narrowly applied to grounding rather than general simulation.

Read-first score

Read-first score 39.1, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 28.

Recency 6%

86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,benchmark,dataset,evaluation

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=code,dataset

Topical relevance 29%

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Citation impact 18%

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. cited_by_count=0

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 378.

Keyword Scores

world model

world dynamics prediction

generative world model

world simulator

interactive world model

video world model

model-based reinforcement learning world model

Deep Analysis

Innovations

Reasoning about future spatial states before making grounding decisions, inspired by world models
Spatial-Aware World Model (SA-WM) that distills the scene into a command-aware latent state and rolls out future latent states
Hypergraph-guided decoder that hierarchically fuses latent states with multimodal input to capture higher-order spatial dependencies
DrivePilot dataset with semantic annotations generated by a Retrieval-Augmented Generation (RAG) and Chain-of-Thought (CoT)-prompted LLM pipeline

Methodology

ThinkDeeper uses a Spatial-Aware World Model (SA-WM) that distills the current scene into a command-aware latent state and rolls out a sequence of future latent states. A hypergraph-guided decoder then hierarchically fuses these states with the multimodal input to capture higher-order spatial dependencies for robust localization. The model is evaluated on six benchmarks including Talk2Car, DrivePilot, MoCAD, and RefCOCO/+/g.

Key Results

ThinkDeeper ranks #1 on the Talk2Car leaderboard and surpasses state-of-the-art baselines on DrivePilot, MoCAD, and RefCOCO/+/g benchmarks. It shows strong robustness and efficiency in challenging scenes (long-text, multi-agent, ambiguity) and retains superior performance even when trained on 50% of the data.

Resources

Paper