⌁ Awesome World Model Hub Papers · Datasets · Projects
中文 GitHub
← Back to papers

Toward Memory-Aided World Models: Benchmarking via Spatial Consistency

arXiv 25.5 2025 81.4 benchmark

TLDR

Proposes LoopNav, a Minecraft dataset and benchmark for evaluating spatial consistency in world models using loop-based navigation.

Reasoning

Strengths: Addresses a clear gap in spatial consistency evaluation for world models with a large, open-source dataset and a novel metric. Weaknesses: Limited to a single simulated environment (Minecraft) and does not demonstrate results on downstream tasks or compare with existing methods.

Read-first score

Read-first score 81.4, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 49.

Reproducibility 25%
100

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=True; dataset=True; markers=code,dataset,github

Recency 8%
86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Methodology quality 25%
80

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark,dataset,experiment,result

Topical relevance 42%
70

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Field roles

FrontierMethodology anchorReproducibility anchor

Rank sensitivity

Stability: volatile; rank range: 28.

Keyword Scores

world model
10
interactive world model
8
world simulator
7
video world model
7
generative world model
6
model-based reinforcement learning world model
6
world dynamics prediction
5

Deep Analysis

Innovations

  • LoopNav dataset: 250 hours (20 million frames) of loop-based navigation videos with actions collected from diverse locations in Minecraft
  • Scene Graph Consistency Score (SGCS) to quantify spatial consistency invariant to pixel-level variations
  • Benchmark for evaluating spatial consistency in world models, addressing the gap in existing datasets and benchmarks

Methodology

The authors propose LoopNav, a dataset of 250 hours (20 million frames) of loop-based navigation videos with actions, collected from diverse locations in the open-world environment of Minecraft. They introduce a Scene Graph Consistency Score to quantify spatial consistency while remaining invariant to pixel-level variations. The dataset, benchmark, and code are open-sourced.

Key Results

No experimental results are reported in the abstract; the paper focuses on dataset and benchmark construction.

Resources

PaperCodeData

Tags