LS-Imagine: Open-World Reinforcement Learning over Long Short-Term Imagination
TLDR
LS-Imagine extends imagination horizon in open-world RL using a long short-term world model with goal-conditioned jumpy transitions, improving exploration and long-term feedback.
Reasoning
The paper introduces a novel method to extend imagination horizon via goal-conditioned jumpy transitions and affordance maps, showing strong empirical results on MineDojo. However, the approach relies on single-image zooming, which may limit generalization, and the abstract does not discuss potential limitations or failure cases.
Read-first score
Read-first score 69.6, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 49.
Field roles
Rank sensitivity
Stability: volatile; rank range: 74.
Keyword Scores
Deep Analysis
Innovations
- Extending imagination horizon within a limited number of state transition steps via a long short-term world model
- Simulating goal-conditioned jumpy state transitions to enable long-horizon exploration
- Computing affordance maps by zooming in on specific areas within single images to integrate direct long-term values into behavior learning
Methodology
LS-Imagine builds a long short-term world model that simulates goal-conditioned jumpy state transitions and computes affordance maps by zooming in on specific areas within single images. This allows the agent to integrate direct long-term values into behavior learning, extending the imagination horizon without requiring many transition steps. The method is evaluated in the MineDojo open-world environment against state-of-the-art techniques.
Key Results
LS-Imagine demonstrates significant improvements over state-of-the-art techniques in the MineDojo environment, indicating better exploration efficiency and long-horizon decision-making.
Limitations
- Evaluation is limited to the MineDojo environment; generalization to other open-world domains is not demonstrated.
- The approach relies on goal-conditioned jumpy state transitions, which may be non-trivial to define for tasks without clear subgoals.
- Computing affordance maps by zooming in on specific areas may introduce computational overhead and may not scale well to very high-resolution images.