Multimodal foundation world models for generalist embodied agents

arXiv 24.6 2024 73.6 method

TLDR

GenRL uses multimodal foundation world models to connect VLMs with generative world models for RL, enabling task specification via vision/language and multi-task generalization.

Reasoning

The paper presents a novel integration of vision-language models with generative world models for reinforcement learning, enabling task specification through vision and language without annotations. Strengths include multi-task generalization and data-free policy learning, but the evaluation is limited to simulated locomotion and manipulation domains, and real-world applicability is not demonstrated.

Read-first score

Read-first score 73.6, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 54.

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=True; dataset=False; markers=code,github

Topical relevance 42%

77.1

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Recency 8%

75.1

Uses a gentle age decay so recent papers surface without erasing older foundations. 2024

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark,result

Field roles

Reproducibility anchor

Rank sensitivity

Stability: volatile; rank range: 78.

Keyword Scores

world model

generative world model

model-based reinforcement learning world model

world simulator

world dynamics prediction

interactive world model

video world model

Deep Analysis

Innovations

Connecting foundation vision-language models (VLMs) with generative world models for reinforcement learning without requiring language annotations
GenRL framework that allows task specification through vision and/or language prompts, grounding them in embodied domain dynamics, and learning behaviors in imagination
Data-free policy learning strategy for foundational policy learning using generative world models

Methodology

The paper proposes multimodal-foundation world models that align the representation of foundation VLMs with the latent space of generative world models for RL. The GenRL framework uses these world models to enable task specification via vision and/or language prompts, grounding them in the embodied domain's dynamics, and learning corresponding behaviors through imagination. The approach is evaluated on large-scale multi-task benchmarks in locomotion and manipulation domains.

Key Results

GenRL enables multi-task generalization from language and visual prompts in locomotion and manipulation domains, as assessed through large-scale multi-task benchmarking.

Resources

Paper Code