Reward-free World Models for Online Imitation Learning
TLDR
Proposes reward-free world models for online imitation learning, using latent dynamics and inverse soft-Q learning to achieve expert-level performance on high-dimensional tasks.
Reasoning
The paper presents a novel integration of world models with imitation learning, addressing instability in high-dimensional tasks. Strengths include a clear methodology and diverse benchmarks (DMControl, MyoSuite, ManiSkill2). Weaknesses are the lack of explicit limitations or comparison details in the abstract.
Read-first score
Read-first score 45.3, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 32.
Field roles
Rank sensitivity
Stability: volatile; rank range: 313.
Keyword Scores
Deep Analysis
Innovations
- Reward-free world models for online imitation learning, learning environmental dynamics in latent spaces without reconstruction
- Inverse soft-Q learning objective reformulated in Q-policy space to mitigate instability in reward-policy space optimization
- Use of learned latent dynamics model and planning for control to achieve stable expert-level performance
Methodology
The method learns environmental dynamics entirely in latent spaces without reconstruction, adopting the inverse soft-Q learning objective to reformulate optimization in the Q-policy space. It employs a learned latent dynamics model and planning for control, enabling efficient and accurate modeling of high-dimensional inputs and complex dynamics.
Key Results
The approach consistently achieves stable, expert-level performance on diverse benchmarks including DMControl, MyoSuite, and ManiSkill2, demonstrating superior empirical performance compared to existing online imitation learning methods.