DreamingV2: Reinforcement Learning with Discrete World Models without Reconstruction

arXiv 22.3 2022 46.7 method

TLDR

DreamingV2 combines discrete world models and reconstruction-free contrastive learning for model-based RL, achieving top scores on simulated robot arm tasks.

Reasoning

The paper presents a novel integration of existing methods (DreamerV2 and Dreaming) with clear empirical results on simulated tasks, but lacks real-world validation and generalizability beyond robot arm environments.

Read-first score

Read-first score 46.7, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 37.

Recency 8%

56.5

Uses a gentle age decay so recent papers surface without erasing older foundations. 2022

Topical relevance 42%

52.9

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Field roles

Candidate

Rank sensitivity

Stability: volatile; rank range: 244.

Keyword Scores

model-based reinforcement learning world model

world model

world dynamics prediction

generative world model

world simulator

interactive world model

video world model

Deep Analysis

Innovations

Combining discrete world models (categorical latent representations) from DreamerV2 with a reconstruction-free contrastive learning objective from Dreaming
Adopting both discrete representation and reconstruction-free training in a single model-based reinforcement learning framework

Methodology

DreamingV2 is a model-based reinforcement learning method that learns from pixels using discrete latent states (categorical variables) and a contrastive learning objective instead of autoencoding reconstruction. It is evaluated on five simulated challenging 3D robot arm tasks, with baselines including DreamerV2 and other recent model-based methods without reconstruction, and performance measured by task scores.

Key Results

DreamingV2 achieves the best scores on all five simulated 3D robot arm tasks compared to DreamerV2 and other reconstruction-free model-based methods.

Resources

Paper