World Models with Hints of Large Language Models for Goal Achieving

arXiv 24.6 2024 37 method

TLDR

DLLM integrates LLM-generated hinting subgoals into model-based RL rollouts to improve goal achievement in sparse-reward environments.

Reasoning

The paper proposes a novel integration of LLMs with world models for guided exploration, showing strong empirical gains. However, it is limited to simulated environments and lacks analysis of LLM hint quality or failure cases.

Read-first score

Read-first score 37, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 18.

Recency 8%

75.1

Uses a gentle age decay so recent papers surface without erasing older foundations. 2024

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=experiment

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Topical relevance 42%

25.7

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Field roles

Candidate

Rank sensitivity

Stability: volatile; rank range: 152.

Keyword Scores

world model

model-based reinforcement learning world model

world dynamics prediction

world simulator

generative world model

interactive world model

video world model

Deep Analysis

Innovations

Integrating hinting subgoals from Large Language Models into model rollouts for goal discovery and reaching
Assigning higher intrinsic rewards to samples that align with LLM hints to guide meaningful and efficient exploration

Methodology

DLLM is a multi-modal model-based RL approach that uses LLMs to generate hinting subgoals, which are integrated into model rollouts. The agent assigns higher intrinsic rewards to samples that align with these hints during rollouts, encouraging goal-directed exploration.

Key Results

DLLM outperforms recent methods in HomeGrid, Crafter, and Minecraft by 27.7%, 21.1%, and 9.9% respectively.

Resources

Paper