Generation Models Know Space: Unleashing Implicit 3D Priors for Scene Understanding

arXiv 26.3 2026 59.4 method, application

TLDR

VEGA-3D repurposes a pre-trained video diffusion model as a Latent World Simulator to extract implicit 3D priors for enhancing MLLMs' spatial reasoning without explicit 3D supervision.

Reasoning

The paper presents a novel approach leveraging implicit spatial priors from video generation models, demonstrating strong empirical results across multiple benchmarks. However, it is limited to spatial reasoning tasks and may not generalize to other domains, and the reliance on pre-trained video models could introduce biases.

Read-first score

Read-first score 59.4, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 38.

Recency 5%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Citation impact 15%

95.9

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. citation_normalized_percentile=0.95917885

Reproducibility 15%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=True; dataset=False; markers=code,github

Methodology quality 15%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,benchmark,experiment,metric

Topical relevance 25%

54.3

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Citation velocity 10%

14.2

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=1.00

Graph prestige 15%

5.7

PageRank-style prestige over the local citation graph built from available OpenAlex references. local_pagerank=0.0016

Field roles

FoundationFrontierMethodology anchorReproducibility anchor

Rank sensitivity

Stability: volatile; rank range: 387.

Keyword Scores

world simulator

world model

video world model

generative world model

world dynamics prediction

interactive world model

model-based reinforcement learning world model

Deep Analysis

Innovations

Paradigm shift: leveraging implicit spatial prior within large-scale video generation models for scene understanding
VEGA-3D: a plug-and-play framework that repurposes a pre-trained video diffusion model as a Latent World Simulator
Token-level adaptive gated fusion mechanism to integrate spatiotemporal features with semantic representations without explicit 3D supervision

Methodology

The method uses a pre-trained video diffusion model as a Latent World Simulator, extracting spatiotemporal features from intermediate noise levels. These features are integrated with semantic representations from MLLMs via a token-level adaptive gated fusion mechanism, enabling dense geometric cues without explicit 3D supervision.

Key Results

VEGA-3D outperforms state-of-the-art baselines across 3D scene understanding, spatial reasoning, and embodied manipulation benchmarks, validating that generative priors provide a scalable foundation for physical-world understanding.

Resources

Paper Code