OSCAR: Omni-Embodiment Action-Conditioned World Model for Robotics
TLDR
OSCAR is an action-conditioned video world model that generalizes across robot embodiments for policy evaluation, using a unified skeleton representation and large-scale data.
Reasoning
The paper addresses key challenges in robot world models with a novel data pipeline and conditioning method, showing strong results and correlation with real-world evaluation. However, it relies on finetuning an existing model and lacks details on data curation and limitations.
Read-first score
Read-first score 70.1, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 61.
Field roles
Rank sensitivity
Stability: volatile; rank range: 393.
Keyword Scores
Deep Analysis
Innovations
- Large-scale standardized data pipeline that curates, filters, and deduplicates broad robotics and egocentric human datasets for joint training
- 2D kinematic skeleton rendering as a unified conditioning representation that generalizes across different robot arms and human hands
- Finetuning the Cosmos-Predict2.5-2B model on a single GH200 GPU, achieving strong performance despite smaller model size and fewer GPUs
- Demonstrating significant correlation between virtual policy evaluation in OSCAR and real-world evaluation using RoboArena
Methodology
OSCAR uses a large-scale standardized data pipeline to curate, filter, and deduplicate diverse robotics and egocentric human datasets. It conditions the video world model via 2D kinematic skeleton rendering, which serves as a unified representation across embodiments. The Cosmos-Predict2.5-2B model is finetuned on a single GH200 GPU, and evaluation is performed against baselines (larger models or more GPUs) and by deploying OSCAR to evaluate robot policies from RoboArena, measuring correlation with real-world outcomes.
Key Results
OSCAR achieves significant improvement on action following, appearance quality, and motion consistency compared to existing baselines. Virtual policy evaluation in OSCAR shows significant correlation with real-world evaluation, indicating its potential for replacing physical robot testing.
Limitations
- Limited scenario diversity in current robot training datasets remains a challenge that the data pipeline only partially addresses
- Imprecise action following is improved but may still have residual errors
- Poor generalization across embodiments is mitigated but not fully solved, especially for unseen robot morphologies
- The model is finetuned on a single GPU, which may limit scalability or reproducibility on different hardware
- Correlation with real-world evaluation is significant but not perfect, leaving uncertainty in fully replacing physical evaluation