Hieros: Hierarchical Imagination on Structured State Space Sequence World Models
TLDR
Hieros uses hierarchical imagination with S5-based world models for sample-efficient deep reinforcement learning, outperforming SOTA on Atari 100k.
Reasoning
The paper introduces a novel hierarchical policy with S5 layer world models that enable parallel training and iterative imagination, achieving strong results on Atari 100k. However, it lacks real-world validation and does not discuss limitations or failure cases.
Read-first score
Read-first score 57.2, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 52.
Field roles
Rank sensitivity
Stability: volatile; rank range: 507.
Keyword Scores
Deep Analysis
Innovations
- Hierarchical policy with time abstracted world representations for imagining trajectories at multiple time scales in latent space
- S5 layer-based world model enabling parallel training and iterative prediction during imagination
- Improved sample efficiency and exploration capabilities over existing world model approaches
Methodology
Hieros uses an S5 layer-based world model that predicts next world states in parallel during training and iteratively during environment interaction. It employs a hierarchical policy that learns time abstracted world representations and imagines trajectories at multiple time scales in latent space. The method is evaluated on the Atari 100k benchmark against state-of-the-art DRL algorithms.
Key Results
Hieros outperforms the state of the art in mean and median normalized human score on the Atari 100k benchmark, and demonstrates superior exploration capabilities and accurate prediction of complex dynamics.