PIN-WM: Learning Physics-INformed World Models for Non-Prehensile Manipulation

arXiv 25.4 2025 58.9 method

TLDR

PIN-WM learns a physics-informed world model from few visual observations for non-prehensile manipulation, using differentiable physics and digital cousins for Sim2Real transfer.

Reasoning

The paper presents a novel integration of differentiable physics with Gaussian Splatting for world model learning, demonstrating strong empirical results in both simulation and real-world tasks. However, the abstract lacks details on the scalability of the approach to more complex scenes or the computational cost of the differentiable physics simulation.

Read-first score

Read-first score 58.9, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 49.

Recency 8%

86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Topical relevance 42%

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=evaluation

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Field roles

Frontier

Rank sensitivity

Stability: volatile; rank range: 364.

Keyword Scores

world model

model-based reinforcement learning world model

world dynamics prediction

interactive world model

world simulator

generative world model

video world model

Deep Analysis

Innovations

Proposes PIN-WM, a physics-informed world model that integrates differentiable physics simulation with Gaussian Splatting for end-to-end identification of 3D rigid body dynamics from visual observations.
Introduces Digital Cousins via physics-aware randomizations to bridge Sim2Real gaps by perturbing physics and rendering parameters, generating diverse variations of the learned world model.
Achieves few-shot and task-agnostic learning of non-prehensile manipulation skills without requiring state estimation, using only observational loss from Gaussian Splatting.

Methodology

PIN-WM learns a world model of 3D rigid body dynamics by combining differentiable physics simulation with Gaussian Splatting for observational loss, enabling end-to-end identification from visual observations. The model is trained with few-shot, task-agnostic physical interaction trajectories and then used for model-based reinforcement learning. To address Sim2Real transfer, the learned world model is transformed into a group of Digital Cousins through physics-aware randomizations that perturb physics and rendering parameters.

Key Results

Extensive evaluations on simulation and real-world tests show that PIN-WM, enhanced with physics-aware digital cousins, facilitates learning robust non-prehensile manipulation skills with Sim2Real transfer, surpassing the Real2Sim2Real state-of-the-art methods.

Limitations

Relies on differentiable physics simulation, which may not accurately model all real-world physical phenomena (e.g., complex friction or restitution).
Gaussian Splatting-based observational loss may struggle with highly cluttered or transparent objects, limiting visual observation quality.
Few-shot learning from task-agnostic trajectories may not generalize to manipulation tasks requiring precise contact or novel object geometries.

Resources

Paper