WorldAgen: Unified State-Action Prediction with Test-Time World Model Training
TLDR
WorldAgen combines world modeling and action prediction in a VLA transformer with test-time training, adapting to new dynamics and improving performance on CALVIN and LIBERO benchmarks.
Reasoning
The paper proposes a clear unified architecture with separate world-model and agent heads, supported by test-time adaptation and benchmark evaluations. Weaknesses include limited evidence of real-world deployment and no details on scalability or failure cases in the abstract.
Read-first score
Read-first score 43.9, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 43.
Field roles
Rank sensitivity
Stability: volatile; rank range: 682.
Keyword Scores
Deep Analysis
Innovations
- Unified framework that jointly learns world modeling and action prediction in a shared Transformer backbone.
- Two-head design: a world model head predicts future states from past state-action trajectories, and an agent model head predicts actions conditioned on task instructions.
- Mixed Unidirectional Attention Mask to separate the world model and agent model computations.
- Test-Time Training (TTT) adaptation mechanism that samples exploratory actions, collects ground-truth state transitions, and applies lightweight updates to the world model at deployment.
Methodology
WorldAgen uses a shared Transformer backbone with two heads: a world model head that predicts future states from past state-action trajectories and an agent model head that predicts actions conditioned on task instructions, separated by a Mixed Unidirectional Attention Mask. During test time, it samples exploratory actions, collects ground-truth state transitions, and performs lightweight Test-Time Training updates to refine the world model. Evaluation is conducted on the CALVIN and LIBERO benchmarks against current state-of-the-art approaches.
Key Results
The baseline WorldAgen model achieves comparable, and in some cases superior, performance to current state-of-the-art approaches on CALVIN and LIBERO. With Test-Time Training on a small number of samples, WorldAgen surpasses existing state-of-the-art models.