World Models

NIPS 2018 Oral 2018 52.5 method

TLDR

Generative neural network models of RL environments enable unsupervised learning and agent training in hallucinated dreams.

Reasoning

Strengths include unsupervised learning and dream training for efficient policy transfer. Weaknesses: limited to simulated environments, no real-world validation.

Read-first score

Read-first score 52.5, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 51.

Topical relevance 42%

72.9

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=none

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=github

Recency 8%

31.9

Uses a gentle age decay so recent papers surface without erasing older foundations. 2018

Field roles

Candidate

Rank sensitivity

Stability: volatile; rank range: 554.

Keyword Scores

world model

generative world model

model-based reinforcement learning world model

world simulator

world dynamics prediction

interactive world model

video world model

Deep Analysis

Innovations

Unsupervised learning of a compressed spatial and temporal representation of reinforcement learning environments using generative neural network models
Training a compact and simple policy using features extracted from the world model
Training an agent entirely inside its own hallucinated dream generated by the world model and transferring the policy back to the actual environment

Methodology

The paper proposes building generative neural network models of popular reinforcement learning environments. The world model is trained quickly in an unsupervised manner to learn a compressed spatial and temporal representation. Features extracted from this world model are then used as inputs to train a compact and simple policy agent.

Key Results

The approach enables training a very compact and simple policy that can solve the required task. The agent can even be trained entirely inside its own hallucinated dream generated by the world model, and the learned policy successfully transfers back to the actual environment.

Resources

Paper