⌁ Awesome World Model Hub Papers · Datasets · Projects
中文 GitHub
← Back to papers

Genie: Generative Interactive Environments

DeepMind 2024 65.6 method, system

TLDR

Genie is a 11B parameter generative interactive world model trained unsupervised from internet videos, enabling action-controllable virtual worlds and imitation learning.

Reasoning

The paper introduces a novel unsupervised approach to training a generative interactive environment from internet videos, eliminating the need for action labels. Its strengths include scalability and the ability to generate diverse virtual worlds from various prompts. However, the abstract lacks quantitative evaluation results and discussion of limitations, making it difficult to assess practical performance.

Read-first score

Read-first score 65.6, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 66.

Topical relevance 42%
94.3

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Recency 8%
75.1

Uses a gentle age decay so recent papers surface without erasing older foundations. 2024

Methodology quality 25%
50

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=result

Reproducibility 25%
30

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Field roles

Candidate

Rank sensitivity

Stability: volatile; rank range: 580.

Keyword Scores

world model
10
generative world model
10
interactive world model
10
video world model
10
world dynamics prediction
10
world simulator
9
model-based reinforcement learning world model
7

Deep Analysis

Innovations

  • First generative interactive environment trained in an unsupervised manner from unlabelled Internet videos
  • Model can be prompted with text, synthetic images, photographs, and sketches to generate action-controllable virtual worlds
  • 11B parameter foundation world model
  • Comprised of a spatiotemporal video tokenizer, an autoregressive dynamics model, and a scalable latent action model
  • Enables frame-by-frame user action control without any ground-truth action labels
  • Learned latent action space facilitates imitation learning from unseen videos

Methodology

Genie consists of a spatiotemporal video tokenizer, an autoregressive dynamics model, and a latent action model. It is trained in an unsupervised manner on unlabelled Internet videos without any ground-truth action labels or domain-specific requirements. The model can be prompted with text, synthetic images, photographs, or sketches to generate action-controllable virtual worlds.

Key Results

Genie enables users to act in generated environments on a frame-by-frame basis despite training without any ground-truth action labels. The learned latent action space facilitates training agents to imitate behaviors from unseen videos, opening the path for training generalist agents.

Resources

PaperBlog

Tags