VisualPatchWorld: Code World Models as Latent Structured Representations for Planning

arXiv 2026 46.1 method

TLDR

VisualPatchWorld represents world dynamics as code, using active probes and parameter fitting for planning, outperforming prior code-based models.

Reasoning

The paper introduces a novel code-based world model with strong empirical results (69.0% planning success, +23.5 points over baseline). Strengths include inspectability and editability; weaknesses are the lack of explicit real-world evaluation and potential limitations in contact-rich tasks.

Read-first score

Read-first score 46.1, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 47.

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,result

Topical relevance 29%

67.1

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=code,github

Citation impact 18%

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. cited_by_count=0

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 447.

Keyword Scores

world model

world dynamics prediction

interactive world model

model-based reinforcement learning world model

world simulator

generative world model

video world model

Deep Analysis

Innovations

Represents world dynamics as code, making models inspectable, editable, and directly usable as simulators.
Selects a qualitative dynamical form via short active probes before fitting free parameters from state-action traces.
Fits code-based model parameters by minimizing multi-step prediction error on recorded trajectories.
Integrates image-derived scene graphs to supply live state for model-predictive control replanning.

Methodology

VisualPatchWorld first uses short active probes to choose a qualitative dynamical form, then optimizes the form's free parameters using recorded state-action traces by minimizing multi-step prediction error. The resulting code programs can be rolled forward as a simulator and used inside model-predictive control, with image-derived scene graphs providing the live state at replanning time.

Key Results

VPW achieves 69.0% mean planning success, exceeding the strongest code baseline by 23.5 points. Under the same planner, induced models approach ground-truth engine success on navigation and grasp-rich control, while a residual gap remains for contact-rich pushing that is mostly closed by checking a shortlist of plans in the engine.

Limitations

A residual performance gap remains for contact-rich pushing tasks, indicating that the automatically constructed code models may not fully capture complex contact dynamics.
Closing the gap on pushing tasks requires additional engine-based verification of promising plans, adding an extra step.

Resources

Paper