Synthesizing world models for bilevel planning

arXiv 25.3 2025 52.6 method

TLDR

TheoryCoder synthesizes hierarchical world models via program synthesis for bilevel planning in grid-world games, improving sample efficiency.

Reasoning

Strengths include a novel integration of program synthesis with hierarchical world models for planning, addressing limitations of prior TBRL systems. Weaknesses are the restriction to grid-world domains and lack of real-world evaluation, leaving scalability and generality unverified.

Read-first score

Read-first score 52.6, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 35.

Recency 8%

86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=ablation,baseline

Topical relevance 42%

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=code

Field roles

Frontier

Rank sensitivity

Stability: volatile; rank range: 322.

Keyword Scores

world model

model-based reinforcement learning world model

world simulator

world dynamics prediction

generative world model

interactive world model

video world model

Deep Analysis

Innovations

TheoryCoder, an instantiation of TBRL that uses hierarchical representations of theories and efficient program synthesis methods
Grounding general-purpose abstractions (e.g., 'move to') by learning low-level transition models as Python programs synthesized from observations by a large language model
A bilevel planning algorithm that exploits hierarchical structure to solve large domains

Methodology

TheoryCoder equips agents with hierarchical abstractions (e.g., 'move to') that are grounded in a particular environment by learning a low-level transition model, which is a Python program synthesized from observations by a large language model. A bilevel planning algorithm then exploits this hierarchical structure to perform planning in large domains. The approach is evaluated on diverse and challenging grid-world games, with baselines including direct policy synthesis.

Key Results

TheoryCoder is successfully applied to diverse and challenging grid-world games, where approaches based on directly synthesizing a policy perform poorly. Ablation studies demonstrate the benefits of using hierarchical abstractions.

Resources

Paper