VILP: Imitation Learning With Latent Video Planning
TLDR
Introduces VILP, an imitation learning method using a latent video diffusion model to generate predictive multi-view robot videos, improving temporal consistency and policy performance while reducing action data reliance.
Reasoning
The paper proposes a practical integration of video generation into robot policy learning, with clear claims about temporal consistency, efficiency, and reduced action data dependence. However, the abstract lacks details about whether evaluations are in simulation or on real robots, and does not specify baselines or limitations.
Read-first score
Read-first score 50.6, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 35.
Field roles
Rank sensitivity
Stability: volatile; rank range: 452.
Keyword Scores
Deep Analysis
Innovations
- Introduces imitation learning with latent video planning (VILP).
- Proposes a latent video diffusion model that generates predictive robot videos with temporal consistency.
- Generates highly time-aligned multi-view videos, including two distinct views at 5 Hz for six 96×160 frames each.
- Demonstrates reduced reliance on extensive high-quality task-specific robot action data and robust representation of multi-modal action distributions.
Methodology
VILP uses a latent video diffusion model to generate predictive robot videos from multiple views for robot policy learning. The approach is evaluated against existing video generation robot policies and other imitation learning methods, with comparisons covering training costs, inference speed, temporal consistency, and policy performance.
Key Results
VILP outperforms existing video generation robot policies on training cost, inference speed, temporal consistency of generated videos, and policy performance. It also maintains robust performance with less extensive high-quality task-specific robot action data and represents multi-modal action distributions robustly.