Learning To Explore With Predictive World Model Via Self-Supervised Learning
TLDR
Proposes an intrinsically motivated agent with a predictive world model via self-supervised learning, achieving superior performance on 18 Atari games.
Reasoning
The paper introduces a novel approach combining cognitive elements and self-supervised learning for world model-based exploration, with strong empirical results on Atari games. However, it lacks real-world validation and generalizability beyond simulated environments.
Read-first score
Read-first score 58.4, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 44.
Field roles
Rank sensitivity
Stability: volatile; rank range: 272.
Keyword Scores
Deep Analysis
Innovations
- Proposes using several neglected cognitive elements to build an internal world model for intrinsically motivated agents
- Employs self-supervised learning to enable exploration without hand-designed reward functions
- Demonstrates emergence of cognitive skills (reactive and deliberative behaviors) in Atari games
Methodology
The methodology involves constructing an internal world model from cognitive elements and training an agent via self-supervised learning to generate intrinsic rewards. The agent is evaluated on 18 Atari games, and its performance is compared against state-of-the-art methods in both dense and sparse reward settings.
Key Results
The agent achieves superior performance compared to state-of-the-art in many test cases across dense and sparse reward environments, and cognitive skills emerge in games requiring reactive and deliberative behaviors.
Limitations
- Evaluation is limited to 18 Atari games, which may not generalize to more complex or diverse environments
- The specific cognitive elements used are not detailed in the abstract, leaving the approach partially unspecified
- The claim of superior performance is qualified as 'in many test cases', indicating inconsistent improvements across all scenarios