DC-MPC: Discrete Codebook World Models for Continuous Control
TLDR
DC-MPC uses discrete codebook world models for continuous control, outperforming continuous latent methods on benchmarks.
Reasoning
The paper presents a novel discrete codebook world model for continuous control, showing competitive results against state-of-the-art methods. Strengths include a clear methodology and empirical validation on benchmarks; weaknesses are the lack of real-world experiments and limited scope to state-based tasks.
Read-first score
Read-first score 63.8, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 40.
Field roles
Rank sensitivity
Stability: volatile; rank range: 247.
Keyword Scores
Deep Analysis
Innovations
- Demonstrating that modeling discrete latent states has benefits over continuous latent states in state-based continuous control
- Showing that discrete codebook encodings are more effective representations for continuous control compared to one-hot and label-based encodings
- Introducing DCWM: a self-supervised world model with a discrete and stochastic latent space using a codebook
- Proposing DC-MPC: a model-based RL algorithm that combines DCWM with decision-time planning
Methodology
The paper proposes DCWM, a self-supervised world model with a discrete and stochastic latent space where latent states are codes from a codebook. This world model is combined with decision-time planning (model predictive control) to form the DC-MPC algorithm. The method is evaluated on state-based continuous control benchmarks, comparing against TD-MPC2 and DreamerV3.
Key Results
DC-MPC performs competitively against recent state-of-the-art algorithms, including TD-MPC2 and DreamerV3, on continuous control benchmarks.