Occ-LLM: Enhancing Autonomous Driving with Occupancy-Based Large Language Models
TLDR
Large Language Models (LLMs) have made substantial advancements in the field of robotic and autonomous driving.
Reasoning
Fallback reasoning generated from available title and abstract metadata: Large Language Models (LLMs) have made substantial advancements in the field of robotic and autonomous driving. This study presents the first Occupancy-based Large Language Model (Occ-LLM), which represents a pioneering effort to integrate LLMs with an important representation....
Read-first score
Read-first score 33.5, weighted from topical fit, citation, graph, method, reproducibility, and recency signals.
Field roles
Rank sensitivity
Stability: volatile; rank range: 125.
Deep Analysis
Innovations
- First integration of Large Language Models with occupancy representation for autonomous driving
- Motion Separation Variational Autoencoder (MS-VAE) that uses prior knowledge to separate dynamic objects from static scenes before encoding
Methodology
Occ-LLM integrates LLMs with occupancy representation using a Motion Separation Variational Autoencoder (MS-VAE) that separates dynamic objects from static scenes to improve encoding and handle category imbalances. The model is evaluated on 4D occupancy forecasting, self-ego planning, and occupancy-based scene question answering tasks.
Key Results
Occ-LLM achieves about 6% improvement in Intersection over Union (IoU) and 4% in mean Intersection over Union (mIoU) for 4D occupancy forecasting over state-of-the-art methods.