CarFormer: Self-Driving with Learned Object-Centric Representations
TLDR
Proposes object-centric slot representations in BEV for self-driving, using a transformer to drive and predict future slots, outperforming baselines.
Reasoning
Strengths include novel object-centric representation using slot attention on BEV sequences, demonstrated improvement over scene-level and object-level baselines, and validation as a world model via forecasting. Weaknesses are reliance on BEV representation without clear generalization to other modalities, and world model validation limited to slot forecasting rather than full dynamics.
Read-first score
Read-first score 56.5, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 21.
Field roles
Rank sensitivity
Stability: volatile; rank range: 594.
Keyword Scores
Deep Analysis
Innovations
- Learning object-centric representations in bird's eye view (BEV) using slot attention to distill complex scenes into actionable information for self-driving.
- Training a transformer on slot representations to simultaneously perform driving and reason about future states of other vehicles.
- Slot representations naturally encode spatial and temporal context (position, heading, speed) without requiring explicit attribute inputs.
- Demonstrating that slot-based object-centric representations outperform both scene-level and object-level approaches that use exact attributes.
- Validating the model as a world model through forecasting experiments, showing accurate prediction of future slot representations.
Methodology
The method first applies a slot attention model on BEV sequences to learn object-centric slot representations. These slots are then used as input to a transformer that learns to drive and reason about the future behavior of other vehicles. The model is evaluated on driving metrics (completion rate, driving score, variance) and forecasting accuracy, with comparisons to scene-level and object-level baselines.
Key Results
The slot-based model achieves increased route completion rate and higher driving score with lower variance across multiple runs compared to baselines. It also demonstrates accurate forecasting of future slot representations, validating its world model capability.