Dream to Control: Learning Behaviors by Latent Imagination
TLDR
Dreamer uses a learned world model for latent imagination to efficiently learn behaviors from images, achieving state-of-the-art on 20 visual control tasks.
Reasoning
The paper introduces a novel method for propagating gradients through imagined trajectories in latent space, demonstrating strong data-efficiency and performance. However, it is evaluated only on simulated tasks, lacking real-world validation.
Read-first score
Read-first score 64.5, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 55.
Field roles
Rank sensitivity
Stability: volatile; rank range: 372.
Keyword Scores
Deep Analysis
Innovations
- Learning behaviors purely by latent imagination in a learned world model without requiring real environment interaction for policy updates
- Propagating analytic gradients of learned state values through trajectories imagined in compact latent state space
- Efficient learning of long-horizon tasks from high-dimensional images
Methodology
Dreamer learns a world model from high-dimensional sensory inputs (images) using deep learning. It then learns behaviors by imagining trajectories in the compact latent state space of the world model and propagating analytic gradients of learned state values back through these trajectories to update the policy. The agent is trained purely from latent imagination without requiring real environment interaction for policy updates.
Key Results
On 20 challenging visual control tasks, Dreamer exceeds existing approaches in data-efficiency, computation time, and final performance.