AutoWorldModel-Bench: A State-Centric Benchmark for Automated World-Model Research

arXiv 2026 40.1 benchmark

TLDR

Introduces AutoWorldModel-Bench, a closed-loop benchmark where AI coding agents autonomously improve world-model starters across eight game environments under fixed compute budgets.

Reasoning

The paper presents a novel benchmark and empirical results across 64 agent sessions, which is a strength. However, the abstract lacks details on baseline comparisons and limitations, and the focus is on agent research rather than advancing world-model methods themselves.

Read-first score

Read-first score 40.1, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 36.

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark,evaluation

Topical relevance 29%

51.4

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=code

Citation impact 18%

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. cited_by_count=0

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 307.

Keyword Scores

world model

world dynamics prediction

world simulator

model-based reinforcement learning world model

interactive world model

generative world model

video world model

Deep Analysis

Innovations

AutoWorldModel-Bench: a closed-loop benchmark for evaluating coding agents as autonomous researchers on open-ended world-model improvement
Unified structured-state representation using ground-truth entity state extracted from each game and consumed through a shared tensor format, decoupling dynamics modeling from perception
Evaluation of frontier coding agents on research-style modifications (new objectives, representations, rollout procedures, architectural changes) rather than engineering-to-spec tasks

Methodology

The benchmark provides a world-model starter and a fixed compute budget, requiring frontier coding agents to autonomously improve it across eight game environments. All environments share a structured-state representation—ground-truth entity state in a common tensor format—so that only dynamics modeling is varied, not perception. The setup runs minutes-per-iteration and was tested in 64 sessions with Codex-5.4 and Claude Opus 4.6.

Key Results

In 63 out of 64 sessions, agents improved the provided starter; 91% of the winning edits were non-trivial research-style modifications rather than simple hyperparameter adjustments.

Limitations

Limited to eight game environments with ground-truth entity state, excluding perception and real-world sensory challenges
Only two frontier coding agents were tested (Codex-5.4 and Claude Opus 4.6) over a total of 64 sessions

Resources

Paper