Reinforcing VLAs in Task-Agnostic World Models
TLDR
Proposes RAW-Dream, a task-agnostic world model and VLM reward for zero-shot VLA fine-tuning via RL, validated in simulation and real-world.
Reasoning
The paper introduces a novel paradigm that decouples world model learning from downstream tasks, using pre-trained task-free world models and off-the-shelf VLMs for reward, enabling zero-shot adaptation. Strengths include real-world experiments and a dual-noise verification mechanism; weaknesses are not explicitly discussed in the abstract, but the approach appears sound.
Read-first score
Read-first score 50.8, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 41.
Field roles
Rank sensitivity
Stability: volatile; rank range: 305.
Keyword Scores
Deep Analysis
Innovations
- Completely disentangling world model learning from downstream task dependencies, enabling task-agnostic world and reward models.
- Using a world model pre-trained on diverse task-free behaviors for predicting future rollouts without task-specific data.
- Leveraging an off-the-shelf Vision-Language Model (VLM) for zero-shot reward generation.
- Introducing a dual-noise verification mechanism to filter out unreliable rollouts and mitigate world model hallucinations.
Methodology
RAW-Dream proposes a paradigm where a world model pre-trained on diverse task-free behaviors generates imagined trajectories, and an off-the-shelf VLM provides rewards without task-specific fine-tuning. VLAs are then fine-tuned entirely within this zero-shot imagination, and a dual-noise verification mechanism filters unreliable rollouts to reduce hallucinations.
Key Results
Extensive experiments across simulation and real-world settings demonstrate consistent performance gains, showing that generalized physical priors can effectively substitute for costly task-dependent data.