Other Vehicle Trajectories Are Also Needed: A Driving World Model Unifies Ego-Other Vehicle Trajectories in Video Latant Space
TLDR
Proposes EOT-WM, a driving world model that unifies ego and other vehicle trajectories in video latent space for controllable simulation.
Reasoning
The paper addresses a key limitation of existing driving world models by enabling control over both ego and other vehicle trajectories, using a novel trajectory-injected diffusion Transformer. Strengths include clear methodology and strong results on nuScenes; weaknesses are reliance on a single dataset and lack of real-world deployment evidence.
Read-first score
Read-first score 67.8, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 53.
Field roles
Rank sensitivity
Stability: volatile; rank range: 106.
Keyword Scores
Deep Analysis
Innovations
- Unifying ego and other vehicle trajectories in video latent space for driving world model
- Projecting BEV trajectories to image coordinates for vehicle-trajectory matching via pixel positions
- Spatial-Temporal Variational Auto Encoder to align trajectory videos with driving video latents
- Trajectory-injected diffusion Transformer for controllable video generation guided by ego-other trajectories
- New metric based on control latent similarity to evaluate trajectory controllability
Methodology
The model first projects ego-other vehicle trajectories from BEV space into image coordinates to match each vehicle with its trajectory via pixel positions. Trajectory videos are then encoded using a Spatial-Temporal Variational Auto Encoder to align with driving video latents in a unified visual space. A trajectory-injected diffusion Transformer denoises noisy video latents to generate videos guided by the ego-other vehicle trajectories.
Key Results
On the nuScenes dataset, the proposed model outperforms the state-of-the-art method by 30% in FID and 55% in FVD, and can also predict unseen driving scenes with self-produced trajectories.