Matrix-Game: Interactive World Foundation Model

arXiv 2025 69.8 method

TLDR

Matrix-Game is an interactive world foundation model for controllable Minecraft generation, trained on 2,700 unlabeled and 1,000 action-labeled hours, outperforming prior world models.

Reasoning

The paper presents a large-scale interactive world model for Minecraft with a two-stage training pipeline, a new annotated dataset, and a comprehensive benchmark. It demonstrates strong empirical gains over prior open-source models and human evaluation. However, scope is limited to Minecraft, and the abstract does not detail architectural innovations or generalization beyond this domain.

Read-first score

Read-first score 69.8, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 48.

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark,dataset,evaluation,experiment,metric

Recency 8%

86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Topical relevance 42%

68.6

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=dataset,github

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 54.

Keyword Scores

interactive world model

world model

generative world model

video world model

world simulator

world dynamics prediction

model-based reinforcement learning world model

Deep Analysis

Innovations

Two-stage training pipeline with large-scale unlabeled pretraining for environment understanding followed by action-labeled training for interactive video generation.
Matrix-Game-MC dataset comprising over 2,700 hours of unlabeled gameplay video clips and over 1,000 hours of labeled clips with fine-grained keyboard and mouse action annotations.
Controllable image-to-world generation paradigm conditioned on a reference image, motion context, and user actions.
Interactive world foundation model with over 17 billion parameters enabling precise control over character actions and camera movements.
GameWorld Score benchmark measuring visual quality, temporal quality, action controllability, and physical rule understanding for Minecraft world generation.

Methodology

Matrix-Game is trained using a two-stage pipeline: large-scale unlabeled pretraining for environment understanding, followed by action-labeled training for interactive video generation. It adopts a controllable image-to-world generation paradigm conditioned on a reference image, motion context, and user actions, with over 17 billion parameters. Evaluation uses the GameWorld Score benchmark and double-blind human evaluations, comparing against prior open-source Minecraft world models including Oasis and MineWorld.

Key Results

Matrix-Game consistently outperforms prior open-source Minecraft world models including Oasis and MineWorld across all GameWorld Score metrics, with particularly strong gains in action controllability and physical consistency. Double-blind human evaluations confirm its superiority in generating perceptually realistic and precisely controllable videos across diverse game scenarios.

Resources

Paper