FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Driving

NeurIPS 25 2025 66.9 method, application

TLDR

FSDrive uses visual spatio-temporal chain-of-thought as a world model to generate future driving scenes, improving trajectory accuracy and collision reduction.

Reasoning

The paper introduces a novel visual CoT that bridges the modality gap between perception and planning, with strong empirical results on nuScenes and NAVSIM. However, it lacks discussion of limitations and real-world deployment challenges, and the reliance on synthetic future frames may limit generalizability.

Read-first score

Read-first score 66.9, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 41.

Recency 8%

86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=True; dataset=False; markers=code,github

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=evaluation,result

Topical relevance 42%

58.6

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Field roles

FrontierReproducibility anchor

Rank sensitivity

Stability: volatile; rank range: 232.

Keyword Scores

world model

generative world model

world dynamics prediction

video world model

world simulator

model-based reinforcement learning world model

interactive world model

Deep Analysis

Innovations

Visual spatio-temporal Chain-of-Thought (CoT) that represents spatial structure and temporal evolution in a single future frame for autonomous driving
Unified framework where the same VLA model acts as both a world model (generating future frames) and an inverse-dynamics model (planning trajectories)
Progressive curriculum that first generates structural priors (lane dividers, 3D boxes) before rendering the full scene to enforce physical laws
Expanding the model's vocabulary with visual tokens and jointly optimizing for semantic understanding (VQA) and future-frame prediction

Methodology

FSDrive first operates as a world model to generate a unified future frame that combines a predicted background with explicit, physically-plausible priors such as future lane dividers and 3D object boxes. This imagined scene serves as a visual spatio-temporal Chain-of-Thought (CoT). The same VLA then functions as an inverse-dynamics model to plan trajectories conditioned on current observations and this visual CoT. A unified pre-training paradigm expands the model's vocabulary with visual tokens and jointly optimizes for semantic understanding (VQA) and future-frame prediction, using a progressive curriculum that first generates structural priors before rendering the full scene.

Key Results

Evaluations on nuScenes and NAVSIM show FSDrive improves trajectory accuracy and reduces collisions. It also achieves competitive FID for video generation with a lightweight autoregressive model and advances scene understanding on DriveLM.

Resources

Paper Code