Teaching Video Generators to Remember: Eliciting Dynamic Memory for Out-of-Sight State Evolution

arXiv 2026 55.4 method, benchmark

TLDR

ReMind elicits dynamic memory in video generators via memory-oriented data and curriculum training, achieving state-of-the-art on STEVO-Bench.

Reasoning

The paper introduces a novel framework to address the limitation of video world models freezing hidden states during interruptions, with a comprehensive data mixture and curriculum. Strengths include a clear problem definition and strong empirical results, but the abstract lacks detailed baseline comparisons and explicit limitations.

Read-first score

Read-first score 55.4, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 45.

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Citation impact 18%

85.7

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. citation_normalized_percentile=0.85724824

Topical relevance 29%

64.3

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=evaluation

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=code

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FoundationFrontierBridge

Rank sensitivity

Stability: volatile; rank range: 352.

Keyword Scores

video world model

world model

generative world model

world dynamics prediction

world simulator

interactive world model

model-based reinforcement learning world model

Deep Analysis

Innovations

ReMind framework for eliciting dynamic memory behavior in video diffusion transformers via memory-oriented data, event-aware training, and cache adaptation.
Taxonomy of 100+ dynamic events and a camera-annotated training mixture combining VLM-filtered real videos, generated hard dynamics, synthetic camera loops, and memory-interruption augmentations.
Frame graph representation with protected anchors, degraded intervals, and explicit temporal gaps to structure training clips.
Node-structured curriculum including node-drop, noisy memory, frontier continuation, and reference-cache training to force retrieval across interruptions.
PM-RoPE, a camera-phase RoPE extension that enables spatiotemporal retrieval at a single-attention cost while preserving pretrained pathways.

Methodology

ReMind constructs a training mixture based on a taxonomy of 100+ dynamic events, using VLM-filtered real videos, generated hard dynamics, synthetic camera loops, and memory-interruption augmentations. Each clip is converted into a frame graph with protected anchors, degraded intervals, and temporal gaps. A node-structured curriculum (node-drop, noisy memory, frontier continuation, reference-cache training) forces the model to retrieve relevant past states across interruptions, and PM-RoPE extends RoPE for spatiotemporal retrieval at single-attention cost.

Key Results

ReMind achieves the best overall scores on STEVO-Bench and recovery tasks. General image-to-video evaluations confirm that the curriculum avoids catastrophic forgetting.

Resources

Paper