MemLearner: Learning to Query Context memory for Video World Models

arXiv 2026 42.5 method

TLDR

MemLearner learns to query context memory adaptively for video world models, improving scene consistency under occlusions and dynamics.

Reasoning

The paper directly addresses a key limitation (memory) in video world models with a novel learning-based query method, supported by a new dataset and multi-dataset training. Strengths include leveraging pre-trained priors and strong empirical results; weaknesses are not evident from the abstract alone but the method's scalability and real-world generalization remain to be fully assessed.

Read-first score

Read-first score 42.5, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 46.

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Topical relevance 29%

65.7

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=dataset,experiment

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=dataset

Citation impact 18%

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. cited_by_count=0

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

Frontier

Rank sensitivity

Stability: volatile; rank range: 321.

Keyword Scores

video world model

interactive world model

world model

generative world model

world dynamics prediction

world simulator

model-based reinforcement learning world model

Deep Analysis

Innovations

Learning-based adaptive context query method using query tokens to bridge context and predicted tokens for memory in video world models
Leveraging the video generation model itself for context querying, exploiting pre-trained visual priors without training additional modules from scratch
Efficient training and inference strategies for the query mechanism
Collection of a new dataset of long videos with scene occlusions and dynamic objects, paired with camera pose annotations
Multi-dataset training strategy combining annotated rendered videos and unannotated real-world videos

Methodology

MemLearner introduces a learning-based adaptive context query approach that uses query tokens within a video generation model to retrieve relevant context frames from memory. It avoids training new modules from scratch by repurposing the generation model's own pre-trained visual priors for querying, and employs efficient training and inference strategies. The method is trained on a newly collected dataset of long videos featuring occlusions and dynamic objects with camera poses, using a multi-dataset strategy that leverages both annotated rendered and unannotated real-world videos.

Key Results

MemLearner significantly outperforms prior video world models in scene consistency and memory, particularly in challenging scenarios with occlusions and dynamic objects.

Resources

Paper