Robust Dreamer: Deviation-Aware Latent Gaussian Memory for Action-Controlled AR Video Generation

arXiv 2026 60.8 method

TLDR

Robust Dreamer introduces Latent Gaussian Memory and Deviation Learning to address drift in action-controlled video generation for interactive world simulation.

Reasoning

The paper presents a novel memory-augmented framework that tackles catastrophic drift from VAE cycling and training-inference gap. However, it lacks explicit real-world experimental validation and does not mention benchmarks or empirical evaluations.

Read-first score

Read-first score 60.8, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 55.

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Citation impact 18%

92.2

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. citation_normalized_percentile=0.9223239

Topical relevance 29%

78.6

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=experiment

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=code

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FoundationFrontierBridge

Rank sensitivity

Stability: volatile; rank range: 496.

Keyword Scores

world simulator

interactive world model

generative world model

video world model

world model

world dynamics prediction

model-based reinforcement learning world model

Deep Analysis

Innovations

Latent Gaussian Memory: anchors diffusion latents to Gaussian primitives and recalls them via latent-space Gaussian splatting, avoiding accumulated degradation from repeated VAE conversion.
Deviation Learning with Dynamic Deviation Archive: synthesizes rollout-induced latent deviations via one-step approximation, stores them by autoregressive stage and denoising timestamp, and injects into historical memory during training to bridge the training-inference gap.

Methodology

Robust Dreamer is a memory-augmented framework for action-controlled image-to-video generation. It introduces Latent Gaussian Memory to provide dense, geometry-aware, view-aligned conditioning by anchoring diffusion latents to Gaussian primitives and recalling them via latent-space Gaussian splatting, avoiding Latent-RGB Cycling degradation. It also proposes Deviation Learning with Dynamic Deviation Archive to synthesize realistic corrupted memory states during training, teaching the generator to correct deviations before inference.

Key Results

Experiments on ScanNet, DL3DV, and OmniWorldGame demonstrate state-of-the-art long-horizon performance in maintaining visual fidelity and 3D consistency over autoregressive rollouts.

Resources

Paper