MemoBench: Benchmarking World Modeling in Dynamically Changing Environments

arXiv 2026 39.9 benchmark

TLDR

MemoBench benchmarks world modeling in dynamic environments using a disappear-and-reappear paradigm with synthetic and real-world clips.

Reasoning

The paper addresses a clear gap in existing benchmarks by evaluating memory consistency under occlusion with dynamic changes, using both automated and VQA metrics. Strengths include a focused diagnostic design and evaluation of eight models; weaknesses include a narrow paradigm that may not generalize to all world modeling tasks.

Read-first score

Read-first score 39.9, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 39.

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark,evaluation,metric

Topical relevance 29%

55.7

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Citation impact 18%

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. cited_by_count=0

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 219.

Keyword Scores

world model

world dynamics prediction

generative world model

video world model

world simulator

interactive world model

model-based reinforcement learning world model

Deep Analysis

Innovations

Introduces the disappear-and-reappear paradigm for evaluating world models in dynamically changing environments, where objects undergo physical processes during occlusion.
Designs a diagnostic benchmark with four evaluation pillars combining automated metrics and VQA-based assessment.
Curates 360 ground-truth clips spanning both synthetic and real-world scenes.

Methodology

MemoBench curates 360 ground-truth video clips covering synthetic and real-world scenes, each featuring a target object that undergoes a physical process, disappears from view, and must be recovered in its updated state. The evaluation suite combines automated metrics with VQA-based assessment across four diagnostic pillars, and eight state-of-the-art video generation models are evaluated.

Key Results

Evaluation of eight state-of-the-art models reveals key insights and open challenges regarding memory consistency under the disappear-and-reappear paradigm.

Resources

Paper