World Model-based Perception for Visual Legged Locomotion
TLDR
Proposes World Model-based Perception for legged locomotion, building a world model to learn policy, outperforming baselines in simulation and real-world.
Reasoning
Strengths: novel integration of world model for visual legged locomotion, demonstrated real-world transfer. Weaknesses: limited detail on world model architecture and training; reliance on simulation may limit generalization.
Read-first score
Read-first score 57.3, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 37.
Field roles
Rank sensitivity
Stability: volatile; rank range: 197.
Keyword Scores
Deep Analysis
Innovations
- World model-based perception for visual legged locomotion without requiring privileged information
- Policy learned from world model predictions rather than through teacher-student imitation, addressing the information gap
- Sim-to-real transfer of a world model that accurately predicts real-world trajectories despite being trained entirely in simulation
Methodology
The proposed World Model-based Perception (WMP) method builds a world model of the environment from proprioceptive and visual inputs, then learns a policy based on the world model's predictions. Training is conducted entirely in simulation, and the world model provides informative signals for the policy controller, avoiding the need for privileged information or a separate imitation learning stage.
Key Results
WMP outperforms state-of-the-art baselines in traversability and robustness in both simulated and real-world experiments, demonstrating effective sim-to-real transfer of the world model.
Limitations
- The method is trained entirely in simulation, which may introduce a sim-to-real gap despite the demonstrated accurate real-world trajectory predictions.
- The world model's performance may depend on the fidelity and diversity of the simulation environment used during training.