OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving

ECCV 24 2024 64.8 method, application

TLDR

OccWorld learns a 3D occupancy world model for autonomous driving, predicting scene evolution and ego trajectory using a GPT-like transformer.

Reasoning

The paper introduces a novel framework using 3D occupancy for world modeling, which is more expressive and efficient than bounding boxes. Strengths include strong empirical results on nuScenes and no need for instance/map supervision. Weaknesses are the narrow focus on autonomous driving and lack of interactive or reinforcement learning components.

Read-first score

Read-first score 64.8, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 35.

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=True; dataset=False; markers=code,github

Recency 8%

75.1

Uses a gentle age decay so recent papers surface without erasing older foundations. 2024

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark,experiment,result

Topical relevance 42%

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Field roles

Methodology anchorReproducibility anchor

Rank sensitivity

Stability: volatile; rank range: 383.

Keyword Scores

world model

world dynamics prediction

generative world model

world simulator

video world model

model-based reinforcement learning world model

interactive world model

Deep Analysis

Innovations

Learning a world model in 3D occupancy space for autonomous driving, enabling simultaneous prediction of ego car movement and scene evolution.
Reconstruction-based scene tokenizer on 3D occupancy to obtain discrete scene tokens for world modeling.
GPT-like spatial-temporal generative transformer to generate future scene and ego tokens, decoding future occupancy and ego trajectory.
Competitive planning results achieved without using instance and map supervision.

Methodology

OccWorld learns a world model based on 3D occupancy rather than 3D bounding boxes or segmentation maps. It first trains a reconstruction-based scene tokenizer on 3D occupancy to produce discrete scene tokens, then employs a GPT-like spatial-temporal generative transformer to predict subsequent scene and ego tokens, which are decoded into future occupancy and ego trajectory. The model is evaluated on the nuScenes benchmark.

Key Results

Experiments on nuScenes demonstrate that OccWorld effectively models the evolution of driving scenes and achieves competitive planning results without relying on instance or map supervision.

Resources

Paper Code