MAMBA: an Effective World Model Approach for Meta-Reinforcement Learning

ICLR 24 2024 56 method

TLDR

MAMBA uses a world model for meta-reinforcement learning, achieving better sample efficiency and return on benchmarks and higher-dimensional domains.

Reasoning

The paper presents a novel model-based meta-RL approach leveraging world models, demonstrating significant sample efficiency gains (up to 15x) on standard benchmarks and extending to higher-dimensional tasks. Strengths include addressing sample efficiency and requiring minimal hyperparameter tuning, but the abstract lacks explicit discussion of limitations and does not include real-world experiments.

Read-first score

Read-first score 56, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 32.

Recency 8%

75.1

Uses a gentle age decay so recent papers surface without erasing older foundations. 2024

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=True; dataset=False; markers=github

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark

Topical relevance 42%

45.7

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Field roles

Reproducibility anchor

Rank sensitivity

Stability: volatile; rank range: 284.

Keyword Scores

world model

model-based reinforcement learning world model

world dynamics prediction

generative world model

world simulator

interactive world model

video world model

Deep Analysis

Innovations

Proposes MAMBA, a novel model-based meta-RL method that leverages world models to improve sample efficiency and handle higher-dimensional task distributions.
Combines elements from existing state-of-the-art model-based and meta-RL methods to create an effective approach for meta-RL.

Methodology

MAMBA is a model-based approach to meta-reinforcement learning that builds upon existing model-based and meta-RL techniques. It uses a learned world model to enable efficient exploration and adaptation across tasks, and is evaluated on both standard meta-RL benchmarks and higher-dimensional domains.

Key Results

MAMBA achieves up to 15x better sample efficiency than existing meta-RL algorithms on common benchmarks while attaining greater return, and also demonstrates effectiveness on more challenging, higher-dimensional domains with minimal hyperparameter tuning.

Resources

Paper Code