Mem-World: Memory-Augmented Action-Conditioned World Models for Persistent Robot Manipulation
TLDR
Mem-World uses a memory-augmented action-conditioned world model with 4D surfel-indexed memory to generate persistent video rollouts for robot manipulation, improving policy evaluation correlation by 14.5%.
Reasoning
The paper introduces a novel memory mechanism (W-VMem) to address occlusion and motion challenges in world modeling for robot manipulation, with strong quantitative results showing improved policy evaluation correlation. However, the approach is tailored to wrist-camera views and may not generalize to other settings without adaptation.
Read-first score
Read-first score 61.3, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 55.
Field roles
Rank sensitivity
Stability: volatile; rank range: 485.
Keyword Scores
Deep Analysis
Innovations
- Proposes W-VMem, a 4D wrist-view-centered surfel-indexed memory that anchors historical observations to temporally evolving surface elements.
- Geometry-aware retrieval of relevant history frames conditioned on future actions via surfel-based rendering and scoring.
- Memory-augmented multi-view action-conditioned world model for persistent robot manipulation.
Methodology
Mem-World is a memory-augmented multi-view action-conditioned world model. It uses W-VMem, a 4D surfel-indexed memory that stores historical observations anchored to surface elements. During generation, relevant history frames are selected via surfel-based rendering and scoring conditioned on future actions, providing informative and non-redundant context for predicting future views.
Key Results
Mem-World generates persistent rollouts in complex manipulation scenarios, improves Pearson correlation with real-world performance by 14.5% over Ctrl-World, and increases success rates from 58% to 72% on long-horizon tasks through synthetic data generation.