Imitation from Heterogeneous Demonstrations using Grounded Latent-Action World Models
TLDR
A grounded latent-action world model aligns heterogeneous demonstration data for imitation learning, outperforming baselines in simulation and real-world tasks.
Reasoning
The paper introduces a novel method (GLAM) that uses a shared latent action space grounded in prediction to handle heterogeneous data sources, which is a clear strength. Weaknesses include reliance on abstract descriptions without detailed experimental results in the abstract, and potential limitations in scalability or complexity not discussed.
Read-first score
Read-first score 53.7, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 35.
Field roles
Rank sensitivity
Stability: volatile; rank range: 385.
Keyword Scores
Deep Analysis
Innovations
- Grounding action representations in prediction: actions that produce the same effect on the environment share the same representation, regardless of source.
- Grounded latent-action world model (GLAM): a pair of generative models with a shared latent action space across heterogeneous data sources, grounded by predicting future observations consistently.
- Co-training approach that avoids heuristic and hand-engineered alignment techniques for heterogeneous demonstration data.
Methodology
GLAM uses a pair of generative models (world models) to learn a shared latent action space across heterogeneous data sources (with and without action labels). The latent actions are grounded by predicting future observations consistently across sources. Downstream behavioral cloning (BC) policies are trained to map observations to latent actions, which are then decoded back to robot actions.
Key Results
Across five manipulation tasks in simulation and the real world, GLAM-aligned policies achieve an average of +48% improvement in task success rate over BC baselines and prior latent-action methods in a data-scarce setting.