Mastering Atari with Discrete World Models

ICLR 21 2021 66.4 method

TLDR

DreamerV2 uses a discrete world model to achieve human-level performance on Atari by learning behaviors purely from latent predictions.

Reasoning

The paper presents a strong contribution with DreamerV2, the first agent to reach human-level on Atari using a separately trained world model. Its strengths include clear methodology and impressive results on 55 tasks, but it lacks real-world validation as all experiments are in simulation.

Read-first score

Read-first score 66.4, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 53.

Topical relevance 42%

75.7

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=True; dataset=False; markers=github

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark

Recency 8%

Uses a gentle age decay so recent papers surface without erasing older foundations. 2021

Field roles

Reproducibility anchor

Rank sensitivity

Stability: volatile; rank range: 291.

Keyword Scores

world model

model-based reinforcement learning world model

world dynamics prediction

generative world model

world simulator

video world model

interactive world model

Deep Analysis

Innovations

Introduces discrete representations in the world model for reinforcement learning
Learns behaviors purely from predictions in the compact latent space of a separately trained world model
First agent to achieve human-level performance on the full Atari benchmark of 55 tasks using a world model

Methodology

DreamerV2 employs a world model with discrete latent representations, trained separately from the policy. The agent learns behaviors by predicting outcomes entirely within this compact latent space. It is evaluated on the Atari benchmark of 55 tasks using the same computational budget and wall-clock time as top single-GPU agents, reaching 200 million frames.

Key Results

DreamerV2 achieves human-level performance on all 55 Atari tasks and surpasses the final performance of IQN and Rainbow. It also learns accurate world models for continuous-action tasks, solving stand-up and walking for a humanoid robot from pixel inputs.

Resources

Paper Code