Mastering Memory Tasks with World Models

ICLR 24 2024 59.8 method

TLDR

Integrating state space models into MBRL world models (R2I) improves long-term memory and credit assignment, achieving SOTA on memory tasks.

Reasoning

The paper addresses a key limitation of MBRL (long-term dependencies) with a novel integration of SSMs, showing strong empirical results across multiple benchmarks. However, it lacks real-world experiments and the abstract does not detail methodology or limitations beyond memory tasks.

Read-first score

Read-first score 59.8, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 34.

Recency 8%

75.1

Uses a gentle age decay so recent papers surface without erasing older foundations. 2024

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=True; dataset=False; markers=github

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,result

Topical relevance 42%

48.6

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Field roles

Reproducibility anchor

Rank sensitivity

Stability: volatile; rank range: 282.

Keyword Scores

world model

model-based reinforcement learning world model

world dynamics prediction

generative world model

interactive world model

world simulator

video world model

Deep Analysis

Innovations

Integration of state space models (SSMs) into world models of model-based reinforcement learning agents
New method Recall to Imagine (R2I) for enhancing long-term memory and long-horizon credit assignment
Achieving superhuman performance on the complex memory domain of Memory Maze
Faster wall-time convergence compared to the state-of-the-art MBRL method DreamerV3

Methodology

R2I integrates a new family of state space models (SSMs) into the world models of MBRL agents to improve temporal coherence. The method is evaluated on a diverse set of tasks including BSuite, POPGym, Memory Maze, Atari, and DMC, with comparisons to DreamerV3 and other baselines.

Key Results

R2I establishes new state-of-the-art results on challenging memory and credit assignment tasks (BSuite and POPGym), achieves superhuman performance on Memory Maze, and maintains comparable performance on classic RL tasks (Atari and DMC) while converging faster in wall-clock time than DreamerV3.

Resources

Paper Code