Scaling Automatic Research Agents via World Models
TLDR
Proposes World Model RL to overcome environment execution bottlenecks in automatic research agents, using debiasing and denoising to accelerate training by 3-4x while improving performance.
评分理由
The paper identifies a real scalability bottleneck in RL for AutoResearch agents and proposes a principled world-model approach with theoretical convergence guarantees and promising empirical speedups. Weaknesses include an incomplete abstract and limited visibility into benchmark details, task diversity, and comparison baselines.
Read-first 评分解释
综合优先阅读分 57.7，由主题、引用、图谱、方法、可复现性和近期性等信号加权得到。 原始总分保留为 63。
研究版图角色
排序敏感性
稳定性：volatile；排名波动范围：33。
关键词评分
深度分析
创新点
- Identifies a fundamental scaling tension in RL for AutoResearch agents: agent generation shares compute through batching while environment execution requires exclusive sandbox and real machine time, making execution the dominant training bottleneck.
- Proposes World Model RL (WMRL), which replaces environment execution with a world model to remove the execution bottleneck.
- Introduces Online Debiasing to offset bias in world-model rewards.
- Introduces Inverse-Variance Denoising to suppress noise in world-model rewards.
- Provides theoretical proof that Online Debiasing and Inverse-Variance Denoising strictly improve the convergence guarantee.
方法
WMRL post-trains AutoResearch agents with RL by replacing real environment execution with a world model, while Online Debiasing and Inverse-Variance Denoising correct the imperfect world-model rewards. The method is evaluated on various tasks at different agent scales against standard RL baselines, with post-trained 4B and 9B agents tested on held-out benchmarks. WMRL is also transferred to post-training embodied VLA policies to test generality.
关键结果
WMRL accelerates training by 3-4x on various tasks at different agent scales while exceeding standard RL baselines. Post-trained 4B and 9B agents outperform much larger open-weight 48B and 120B agents on held-out benchmarks, and the method transfers to embodied VLA post-training.
局限性
- The world model can be imperfect, with rewards corrupted by bias and noise, requiring additional mitigation via Online Debiasing and Inverse-Variance Denoising.