⌁ Awesome World Model Hub Papers · Datasets · Projects
中文 GitHub
← Back to papers

AGI Maze Prediction Datasets: A Compact Benchmark for Learning World Dynamics with Transformers

arXiv 2026 38.3 benchmark

TLDR

Introduces a synthetic maze benchmark for testing Transformers' world dynamics prediction, showing structured spatial memory outperforms byte-level and unstructured-memory baselines.

Reasoning

The paper provides a controlled benchmark and clear comparisons across architectures, which is a strength. However, it relies on synthetic grid worlds rather than real-world tasks, and the abstract is truncated, limiting full assessment of limitations.

Read-first score

Read-first score 38.3, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 33.

Methodology quality 15%
100

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,benchmark,dataset,evaluation,result,validation

Recency 5%
100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Topical relevance 25%
47.1

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 15%
38

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=dataset

Graph prestige 15%
5.7

PageRank-style prestige over the local citation graph built from available OpenAlex references. local_pagerank=0.0016

Citation impact 15%
0

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. cited_by_count=0

Citation velocity 10%
0

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 721.

Keyword Scores

world dynamics prediction
9
world model
8
interactive world model
5
generative world model
4
world simulator
3
video world model
2
model-based reinforcement learning world model
2

Deep Analysis

Innovations

  • Introduces the AGI Maze Prediction Datasets and Benchmark, a lightweight controlled testbed derived from procedurally generated stateful grid worlds for studying world dynamics learning.
  • Defines three prediction tasks: per-step transition prediction, fixed-horizon state prediction, and sequential textual-observation prediction.
  • Uses source-maze-disjoint training and validation splits with greedy exact-match evaluation to separate transferable action-conditioned dynamics from memorization of familiar mazes.
  • Proposes a pseudo-video spatial-memory Transformer that initializes a two-dimensional latent workspace from the input map and updates it from action history without intermediate maps, positions, or state labels.
  • Shows that structured, task-aligned working memory can outperform additional latent capacity alone, supporting the view that language grounding may be mediated by persistent data structures.

Methodology

The benchmark is built from procedurally generated stateful grid worlds and includes per-step transition prediction, fixed-horizon state prediction, and sequential textual-observation prediction tasks. It uses source-maze-disjoint training and validation splits with greedy exact-match evaluation. The study trains from-scratch byte-level Transformer baselines and compares them with two working-memory-augmented architectures under identical data, objectives, and evaluation protocols.

Key Results

A generic auxiliary latent-memory Transformer can fit some training sets perfectly but does not consistently improve held-out performance. A pseudo-video spatial-memory Transformer achieves perfect validation accuracy on selected fixed-horizon tasks where byte-level and unstructured-memory baselines fail, and substantially improves sequential text-trace prediction.

Limitations

  • Perfect validation accuracy is reported only on selected fixed-horizon tasks, not across all benchmark tasks.
  • The benchmark is lightweight and controlled, so findings may not directly transfer to larger-scale or real-world world-modeling settings.
  • The abstract does not report statistical significance or variance across multiple training runs or maze configurations.

Resources

Paper

Tags