NORA-1.5: A Vision-Language-Action Model Trained using World Model- and Action-based Preference Rewards
TLDR
NORA-1.5 enhances VLA models with flow-matching action expert and world model-based preference rewards, improving reliability in simulation and real-world tasks.
Reasoning
The paper presents a clear architectural improvement and a novel reward framework using an action-conditioned world model, with strong empirical results across both simulated and real-world benchmarks. However, the abstract lacks details on the world model's architecture and comparisons to other world model approaches, limiting depth.
Read-first score
Read-first score 55.1, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 17.
Field roles
Rank sensitivity
Stability: volatile; rank range: 654.
Keyword Scores
Deep Analysis
Innovations
- Adding a flow-matching-based action expert to the pre-trained NORA backbone
- Developing reward models combining an action-conditioned world model and a deviation-from-ground-truth heuristic
- Using direct preference optimization (DPO) with preference datasets constructed from these reward signals for post-training
Methodology
NORA-1.5 is built from the pre-trained NORA backbone by adding a flow-matching-based action expert. For post-training, reward models are developed: an action-conditioned world model that evaluates whether generated actions lead toward the desired goal, and a deviation-from-ground-truth heuristic that distinguishes good actions from poor ones. Preference datasets are constructed using these reward signals, and the model is adapted to target embodiments through direct preference optimization (DPO).
Key Results
NORA-1.5 outperforms NORA and several state-of-the-art VLA models across both simulated and real-world benchmarks. Reward-driven post-training consistently improves performance in both simulation and real-robot settings.