MIND: Benchmarking Memory Consistency and Action Control in World Models

arXiv 26.2 2026 79 benchmark

TLDR

MIND is a benchmark for evaluating memory consistency and action control in world models using 250 high-quality videos across diverse scenes and action spaces.

Reasoning

The paper introduces a novel benchmark (MIND) with a clear evaluation framework and a baseline (MIND-World), addressing a gap in world model evaluation. Strengths include open-domain, closed-loop design and diverse action spaces; weaknesses are limited video count and scene diversity, and lack of detailed results or comparisons in the abstract.

Read-first score

Read-first score 79, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 51.

Recency 8%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=True; dataset=False; markers=code,github

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,benchmark,evaluation,experiment

Topical relevance 42%

72.9

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Field roles

FrontierMethodology anchorReproducibility anchor

Rank sensitivity

Stability: volatile; rank range: 23.

Keyword Scores

world model

world dynamics prediction

interactive world model

video world model

generative world model

model-based reinforcement learning world model

world simulator

Deep Analysis

Innovations

First open-domain closed-loop revisited benchmark for evaluating memory consistency and action control in world models
Efficient evaluation framework for measuring memory consistency and action control, capturing temporal stability and contextual coherence across viewpoints
Design of various action spaces (different character movement speeds and camera rotation angles) to evaluate action generalization capability across different action spaces under shared scenes
Introduction of MIND-World, a novel interactive Video-to-World baseline for benchmarking

Methodology

MIND contains 250 high-quality videos at 1080p and 24 FPS, including 100 first-person and 100 third-person video clips under a shared action space, and 25+25 clips across varied action spaces covering eight diverse scenes. The evaluation framework measures two core abilities: memory consistency and action control. MIND-World is an interactive Video-to-World baseline used for benchmarking.

Key Results

Extensive experiments demonstrate the completeness of MIND and reveal key challenges in current world models, including the difficulty of maintaining long-term memory consistency and generalizing across action spaces.

Resources

Paper Code