MemoryVLA++: Temporal Modeling via Memory and Imagination in Vision-Language-Action Models
TLDR
MemoryVLA++ integrates memory and a world model into VLA models for temporally consistent robotic manipulation.
Reasoning
The paper presents a novel framework combining working memory, episodic memory, and a world model for future state imagination, addressing temporal modeling in robotic tasks. Strengths include extensive experiments on simulation and real robots. Weaknesses are limited architectural details of the world model and potential reliance on pretrained VLMs.
Read-first score
Read-first score 58, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 35.
Field roles
Rank sensitivity
Stability: volatile; rank range: 415.
Keyword Scores
Deep Analysis
Innovations
- Introduces a full temporal modeling framework for VLA models inspired by cognitive science, incorporating working memory, episodic memory via a Perceptual-Cognitive Memory Bank, and imagination through a world model for future state prediction.
- Proposes redundancy-aware consolidation for updating the memory bank, and integrates imagined latents under memory guidance to form temporal-aware tokens for action prediction.
Methodology
MemoryVLA++ uses a pretrained VLM to encode current observations into perceptual and cognitive tokens (working memory). These tokens query a Perceptual-Cognitive Memory Bank that stores low-level details and high-level semantics from past interactions, updated via redundancy-aware consolidation. A world model imagines future states in a denoising latent space, and the imagined latents are integrated under memory guidance to form full temporal-aware tokens, which then condition a diffusion action expert to predict temporally consistent action sequences.
Key Results
The method achieves strong performance across 5 simulation benchmarks (Libero, SimplerEnv, Mikasa-Robo, Calvin, Libero-Plus) and 3 categories of real-robot tasks on 3 robots. On real robots, it obtains +9%, +26%, and +28% gains on general, memory-dependent, and imagination-dependent tasks respectively.