VLAW: Iterative Co-Improvement of Vision-Language-Action Policy and World Model
TLDR
Iteratively improving a vision-language-action policy and a video-based world model using real-world rollouts to generate synthetic data, achieving significant success rate improvements on a real robot.
Reasoning
The paper presents a practical iterative co-improvement framework that leverages real-world rollouts to enhance a video-based world model, which then generates synthetic data to improve a VLA policy. Strengths include addressing real-world data scarcity and demonstrating clear performance gains on a real robot. Weaknesses are the limited discussion of world model fidelity across diverse tasks and potential generalizability issues.
Read-first score
Read-first score 69.9, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 58.
Field roles
Rank sensitivity
Stability: volatile; rank range: 147.
Keyword Scores
Deep Analysis
Innovations
- Iterative co-improvement of vision-language-action (VLA) policy and world model through online interaction
- Using real-world rollout data to improve the fidelity of an action-conditioned video generation world model
- Leveraging the improved world model to generate synthetic rollout data for further VLA policy improvement
Methodology
The paper proposes an iterative algorithm that first collects real-world policy rollouts, uses them to fine-tune an action-conditioned video generation world model to improve its physical fidelity, and then employs the improved world model to generate synthetic rollout data for training the VLA policy. The approach is evaluated on a real robot with multiple downstream tasks, comparing the base VLA policy against the policy trained with synthetic rollouts.
Key Results
The method achieves a 39.2% absolute success rate improvement over the base VLA policy and an additional 11.6% improvement from training with the generated synthetic rollouts on real robot tasks.
Limitations
- The world model may still struggle to accurately model small yet critical physical details in contact-rich object manipulation
- The iterative improvement relies on real-world rollouts, which are expensive to collect, though less so than full policy training
- The approach is only demonstrated on specific downstream tasks, and generalization to broader scenarios is not established
- Potential domain gap between synthetic rollouts generated by the world model and real-world dynamics may limit policy improvement