Unified 3D Scene Understanding Through Physical World Modeling

arXiv 2026 55.6 method

TLDR

A unified physical world model (3WM) for 3D scene understanding, enabling zero-shot depth estimation, novel view synthesis, and object manipulation.

Reasoning

The paper presents a novel unified framework that integrates multiple 3D vision tasks into a single probabilistic graphical model, achieving strong performance without task-specific training. However, it focuses narrowly on static 3D scene tasks and does not address temporal dynamics or reinforcement learning, limiting its scope as a general world model.

Read-first score

Read-first score 55.6, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 37.

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Citation impact 18%

85.7

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. citation_normalized_percentile=0.85726818

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,dataset,metric

Topical relevance 29%

52.9

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=dataset

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FoundationFrontierBridgeMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 295.

Keyword Scores

world model

generative world model

interactive world model

world simulator

video world model

world dynamics prediction

model-based reinforcement learning world model

Deep Analysis

Innovations

Unified model for multiple 3D understanding tasks (depth estimation, novel view synthesis, object manipulation) using a single architecture
Probabilistic graphical model where nodes represent multimodal scene elements (RGB, optical flow, camera pose) enabling different inference pathways for different tasks
Zero-shot task execution without task-specific training, using different prompts (e.g., dense flow, sparse flow, camera conditioning)
Composable inference pathways allowing complex geometric reasoning, such as moving objects aside while navigating a 3D environment

Methodology

The paper proposes 3WM, a physical world model formulated as a probabilistic graphical model. Nodes represent multimodal scene elements including RGB, optical flow, and camera pose. Diverse tasks emerge from different inference pathways through the graph, and the model is trained jointly across all datasets without task-specific objectives.

Key Results

3WM outperforms specialized baselines without finetuning, achieving state-of-the-art performance on novel view synthesis and 3D object manipulation, while offering precise controllability, strong geometric consistency, and robustness in real-world scenarios.

Resources

Paper