Out of Sight but Not Out of Mind: Hybrid Memory for Dynamic Video World Models

arXiv 26.3 2026 69.6 method, benchmark

TLDR

Introduces Hybrid Memory and HyDRA for video world models to maintain subject consistency when hidden, with a new dataset HM-World.

Reasoning

The paper addresses a clear gap in video world models by proposing a hybrid memory paradigm and a specialized architecture (HyDRA) that outperforms existing methods on a new large-scale dataset (HM-World). However, the contribution is limited to video generation tasks and does not extend to interactive or reinforcement learning settings, and the term 'world model' may be overstated for a video prediction context.

Read-first score

Read-first score 69.6, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 42.

Recency 8%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=True; dataset=False; markers=code,dataset,github

Topical relevance 42%

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=dataset,experiment

Field roles

FrontierReproducibility anchor

Rank sensitivity

Stability: volatile; rank range: 220.

Keyword Scores

video world model

world model

world dynamics prediction

generative world model

world simulator

interactive world model

model-based reinforcement learning world model

Deep Analysis

Innovations

Hybrid Memory paradigm requiring models to act as precise archivists for static backgrounds and vigilant trackers for dynamic subjects, ensuring motion continuity during out-of-view intervals.
HM-World, the first large-scale video dataset dedicated to hybrid memory, featuring 59K high-fidelity clips with decoupled camera and subject trajectories, 17 scenes, 49 subjects, and meticulously designed exit-entry events.
HyDRA, a specialized memory architecture that compresses memory into tokens and utilizes a spatiotemporal relevance-driven retrieval mechanism to preserve identity and motion of hidden subjects.

Methodology

The paper proposes HyDRA, a memory architecture that compresses memory into tokens and employs a spatiotemporal relevance-driven retrieval mechanism to selectively attend to relevant motion cues. The method is evaluated on the newly constructed HM-World dataset, which includes 59K clips with decoupled camera and subject trajectories and exit-entry events. The approach is compared against state-of-the-art methods in dynamic video world modeling.

Key Results

Extensive experiments on HM-World demonstrate that HyDRA significantly outperforms state-of-the-art approaches in both dynamic subject consistency and overall generation quality.

Resources

Paper Code Project