TD-MPC2: Scalable, Robust World Models for Continuous Control
TLDR
TD-MPC2 improves model-based RL with scalable, robust world models, achieving strong results across 104 tasks with a single hyperparameter set.
Reasoning
The paper presents clear improvements over TD-MPC with extensive empirical validation across many tasks, demonstrating scalability. However, it lacks real-world physical experiments and does not address generative or video world models explicitly.
Read-first score
Read-first score 68.4, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 41.
Field roles
Rank sensitivity
Stability: volatile; rank range: 221.
Keyword Scores
Deep Analysis
Innovations
- Series of improvements upon the TD-MPC algorithm for model-based RL
- Single set of hyperparameters achieving strong results across 104 diverse online RL tasks
- Demonstration of scaling laws: agent capabilities increase with model and data size
- Successful training of a single 317M parameter agent to perform 80 tasks across multiple domains, embodiments, and action spaces
Methodology
TD-MPC2 is a model-based reinforcement learning algorithm that performs local trajectory optimization in the latent space of a learned implicit (decoder-free) world model. It is evaluated on 104 online RL tasks spanning 4 diverse task domains using a single set of hyperparameters, and scaling experiments are conducted with a 317M parameter agent trained on 80 tasks.
Key Results
TD-MPC2 significantly improves over baselines across all 104 online RL tasks, achieving consistently strong results with a single hyperparameter set. Additionally, agent performance scales with model and data size, culminating in a single 317M parameter agent that successfully performs 80 tasks across multiple domains.