LDA-1B: Scaling Latent Dynamics Action Model via Universal Embodied Data Ingestion
TLDR
LDA-1B scales robot foundation model by jointly learning dynamics, policy, and visual forecasting from diverse embodied data, outperforming prior methods.
Reasoning
The paper introduces a novel approach to scaling robot foundation models via universal data ingestion and latent dynamics prediction, with strong empirical results in simulation and real world. However, the reliance on a specific latent representation (DINO) and complex multi-modal architecture may limit generalizability.
Read-first score
Read-first score 40.8, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 42.
Field roles
Rank sensitivity
Stability: volatile; rank range: 263.
Keyword Scores
Deep Analysis
Innovations
- Universal embodied data ingestion by jointly learning dynamics, policy, and visual representations, assigning distinct roles to data of varying quality.
- Scalable dynamics learning via prediction in a structured DINO latent space, avoiding pixel-space appearance modeling.
- Multi-modal diffusion transformer to handle asynchronous vision and action streams, enabling stable training at the 1B-parameter scale.
- Creation of EI-30k, a unified embodied interaction dataset with over 30k hours of human and robot trajectories.
- Data-efficient fine-tuning that leverages low-quality trajectories (30% typically harmful) to gain 10% improvement.
Methodology
LDA-1B is a 1B-parameter robot foundation model that jointly learns dynamics, policy, and visual representations from a unified heterogeneous dataset (EI-30k). It uses a structured DINO latent space for dynamics prediction to avoid pixel-space modeling and a multi-modal diffusion transformer to process asynchronous vision and action streams.
Key Results
LDA-1B outperforms prior methods like π_{0.5} by up to 21%, 48%, and 23% on contact-rich, dexterous, and long-horizon tasks, respectively, and achieves data-efficient fine-tuning gaining 10% by leveraging 30% low-quality trajectories.