JEDI: Latent End-to-end Diffusion Mitigates Agent-Human Performance Asymmetry in Model-Based Reinforcement Learning
TLDR
Proposes JEDI, a latent diffusion world model to address performance asymmetry in MBRL on Atari100k, achieving balanced human-normalized scores.
Reasoning
Strengths: Identifies a novel performance asymmetry issue in MBRL and proposes a method to mitigate it. Weaknesses: Limited to Atari100k benchmark; generalization unclear.
Read-first score
Read-first score 70.6, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 56.
Field roles
Rank sensitivity
Stability: volatile; rank range: 88.
Keyword Scores
Deep Analysis
Innovations
- Identification and formalization of Performance Asymmetry in MBRL on Atari100k, where agents outperform humans on some tasks and underperform on others
- Introduction of Sym-HNS, a more balanced aggregate metric that evenly weights Human-Optimal and Agent-Optimal subsets
- Proposal of JEDI (Joint Embedding DIffusion) world model, a latent end-to-end diffusion model that operates in latent space to mitigate curse of dimensionality
- Reversing the worsening Performance Asymmetry trend while improving computational efficiency and achieving SOTA on Sym-HNS, Human-Optimal tasks, and Breakout
Methodology
The authors partition the Atari100k benchmark into Human-Optimal and Agent-Optimal subsets based on performance asymmetry. They propose a latent end-to-end joint embedding diffusion (JEDI) world model that learns in a latent space to address the curse of dimensionality inherent in pixel-level diffusion models. The model is evaluated on Atari100k using standard MBRL training and evaluation, with baselines including prior SOTA pixel diffusion world models, and metrics include HNS, Sym-HNS, and per-task performance.
Key Results
JEDI achieves state-of-the-art results on Sym-HNS, Human-Optimal tasks, and the Breakout task, reversing the worsening Performance Asymmetry trend while remaining competitive on the full Atari100k benchmark and improving computational efficiency.