PatchWorld: Gradient-Free Optimization of Executable World Models
TLDR
PatchWorld induces executable Python world models from offline trajectories via counterexample-guided code repair, achieving high planning scores without LLM calls.
Reasoning
The paper introduces a novel gradient-free method for creating symbolic, inspectable world models from offline data, with strong empirical results on AgentGym environments. However, the approach is limited to text-agent domains and reveals a tradeoff between observation fidelity and decision utility that may constrain practical use.
Read-first score
Read-first score 60.3, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 35.
Field roles
Rank sensitivity
Stability: volatile; rank range: 462.
Keyword Scores
Deep Analysis
Innovations
- Gradient-free framework that turns offline trajectories into executable Python world models via counterexample-guided code repair
- Induces symbolic belief-state programs whose action updates can be inspected, replayed, and locally patched
- No LLM calls inside the world-model prediction module itself
Methodology
PatchWorld uses offline trajectories to induce executable Python world models through counterexample-guided code repair. It produces symbolic belief-state programs that can be inspected and patched locally. The method is evaluated on seven AgentGym environments using code-based planning scores and compared against baselines.
Key Results
PatchWorld-Simple achieves the highest code-based planning score among evaluated methods, reaching 76.4% macro success in live one-step lookahead while invoking no LLM calls inside the world-model prediction module.
Limitations
- A human-specified residual-memory bias improves surface observation fidelity but weakens decision utility
- Improving observation fidelity can come at the expense of action-discriminative dynamics, exposing a tradeoff in executable world models