Learning a Particle Dynamics Model with Real-world Videos
TLDR
Learns particle dynamics from real-world videos using Gaussian splatting and rendering supervision, overcoming sim-to-real gap.
Reasoning
Strengths include a novel framework for training on unlabeled real videos and a new dataset of 500 videos. Weaknesses are the focus on particle dynamics rather than a full world model, and potential limitations in complex scenes.
Read-first score
Read-first score 51.3, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 31.
Field roles
Rank sensitivity
Stability: volatile; rank range: 226.
Keyword Scores
Deep Analysis
Innovations
- Training neural object dynamics models directly from unlabeled real-world videos, overcoming the sim-to-real gap
- Particle-based dynamics model compatible with Gaussian splatting, operating on dense particles with scales and rotations
- Training via rendering supervision without requiring particle-level labeled states
- Introduction of a real-world dataset of about 500 videos capturing diverse object interactions
Methodology
The paper proposes a particle-based dynamics model that operates on dense particles derived from Gaussians (with scales and rotations) and predicts their position and rotation changes over time. The model is trained via rendering supervision from unlabeled real-world videos, eliminating the need for particle-level state labels. A dataset of about 500 videos is used for training and evaluation.
Key Results
The model is trained on a real-world dataset of about 500 videos, enabling learning of object dynamics from unlabeled video data. No quantitative results are reported in the abstract.