SparseWorld-TC: Trajectory-Conditioned Sparse Occupancy World Model
TLDR
A transformer-based sparse occupancy world model for trajectory-conditioned future 3D scene occupancy forecasting, achieving SOTA on nuScenes.
Reasoning
The paper presents a novel architecture that avoids VAE tokenization and BEV projection, directly predicting multi-frame occupancy from raw images. Its strength lies in the end-to-end transformer design and strong empirical results on a real-world benchmark. Weaknesses include lack of discussion on generalization beyond nuScenes and no comparison to generative or interactive world models.
Read-first score
Read-first score 34.2, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 28.
Field roles
Rank sensitivity
Stability: volatile; rank range: 130.
Keyword Scores
Deep Analysis
Innovations
- End-to-end prediction of multi-frame future occupancy directly from raw image features, avoiding VAE-based discrete tokenization
- Sparse occupancy representation that bypasses intermediate bird's eye view (BEV) projection and explicit geometric priors
- Attention-based transformer architecture inspired by GPT and VGGT to capture spatiotemporal dependencies effectively
- Trajectory-conditioned forecasting enabling robust scene dynamics understanding under arbitrary future trajectories
Methodology
The method predicts multi-frame future 3D scene occupancy in an end-to-end manner directly from raw image features using a sparse occupancy representation and an attention-based transformer architecture. It avoids both the finite-capacity constraint of discrete tokenization via VAEs and the structural limitations of BEV projection.
Key Results
Achieves state-of-the-art performance on the nuScenes benchmark for 1-3 second occupancy forecasting, outperforming existing approaches by a significant margin, and demonstrates robust scene dynamics understanding under arbitrary future trajectory conditioning.