dWorldEval: Scalable Robotic Policy Evaluation via Discrete Diffusion World Model
TLDR
Proposes dWorldEval, a discrete diffusion world model for scalable robotic policy evaluation using unified token space and progress token.
Reasoning
The paper introduces a novel discrete diffusion world model that unifies vision, language, and actions into tokens, with a progress token for task completion. Strengths include strong empirical results on multiple benchmarks and real-robot tasks. Weaknesses are not explicitly discussed in the abstract, but the approach appears promising for scalable evaluation.
Read-first score
Read-first score 52.1, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 45.
Field roles
Rank sensitivity
Stability: volatile; rank range: 308.
Keyword Scores
Deep Analysis
Innovations
- Mapping all modalities (vision, language, robotic actions) into a unified token space
- Single transformer-based denoising network for discrete diffusion world model
- Sparse keyframe memory to maintain spatiotemporal consistency
- Progress token indicating task completion degree for automatic success detection
Methodology
dWorldEval uses a discrete diffusion world model as a scalable evaluation proxy. It maps vision, language, and robotic actions into a unified token space and denoises them with a single transformer network. A sparse keyframe memory ensures spatiotemporal consistency, and a progress token is introduced to automatically determine task success when its value reaches 1.
Key Results
dWorldEval significantly outperforms previous approaches (WorldEval, Ctrl-World, WorldGym) on LIBERO, RoboTwin, and multiple real-robot tasks.