DSG-World: Learning a 3D Gaussian World Model from Dual State Videos

arXiv 25.6 2025 52.5 method

TLDR

DSG-World builds an explicit 3D Gaussian world model from dual state videos, enabling efficient real-to-sim transfer and object manipulation.

Reasoning

The paper introduces a novel end-to-end framework that leverages dual perturbed observations to overcome occlusion and achieve stable 3D reconstruction. Its strengths include explicit 3D representation and strong generalization, but it lacks discussion of limitations and may rely on the dual-state assumption.

Read-first score

Read-first score 52.5, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 34.

Recency 8%

86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=experiment,metric

Topical relevance 42%

48.6

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 316.

Keyword Scores

world model

generative world model

world simulator

video world model

interactive world model

world dynamics prediction

model-based reinforcement learning world model

Deep Analysis

Innovations

Leveraging dual state videos (two perturbed observations of the same scene under different object configurations) to alleviate occlusion and enable stable 3D reconstruction
End-to-end framework that explicitly constructs a 3D Gaussian world model from dual state observations without multi-stage pipelines
Dual segmentation-aware Gaussian fields with bidirectional photometric and semantic consistency enforcement
Pseudo intermediate state for symmetric alignment between the two Gaussian fields
Collaborative co-pruning strategies to refine geometric completeness
Real-to-simulation transfer purely in the explicit Gaussian representation space, supporting high-fidelity rendering and object-level scene manipulation

Methodology

DSG-World is an end-to-end framework that builds a 3D Gaussian world model from dual state videos. It constructs dual segmentation-aware Gaussian fields, enforces bidirectional photometric and semantic consistency between them, introduces a pseudo intermediate state for symmetric alignment, and applies collaborative co-pruning strategies to refine geometric completeness. The model is trained on dual state observations and evaluated on novel view and scene state generalization.

Key Results

Extensive experiments demonstrate strong generalization to novel views and scene states, highlighting the effectiveness of the approach for real-world 3D reconstruction and simulation without relying on dense observations or multi-stage pipelines.

Limitations

Requires two perturbed observations (dual states) of the same scene, which may not be available in all real-world scenarios
Potential sensitivity to the alignment and consistency between the two states, especially under large occlusions or dynamic changes beyond two configurations

Resources

Paper