ProPlay: Procedural World Models for Self-Evolving LLM Agents

arXiv 2026 64.1 method

TLDR

ProPlay introduces a procedural world model that abstracts trajectories into a graph for preplay simulation, enabling self-evolving LLM agents to improve through interaction.

Reasoning

Strengths: novel procedural abstraction and graph-based world model for self-evolution, with clear methodology and code release. Weaknesses: abstract lacks specific benchmark results and real-world task details, limiting assessment of generalizability.

Read-first score

Read-first score 64.1, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 35.

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Citation impact 18%

95.8

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. citation_normalized_percentile=0.95808064

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=True; dataset=False; markers=code,github

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,benchmark,experiment

Topical relevance 29%

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FoundationFrontierBridgeMethodology anchorReproducibility anchor

Rank sensitivity

Stability: volatile; rank range: 503.

Keyword Scores

world model

world dynamics prediction

world simulator

interactive world model

model-based reinforcement learning world model

generative world model

video world model

Deep Analysis

Innovations

Procedural world model that supports procedure-level preplay for self-evolving LLM agents
Procedure graph that captures causal transitions among task stages with reliability record embeddings
Closing the loop between memory and planning by refining internal understanding of environment dynamics through interaction

Methodology

ProPlay introduces a procedural world model that abstracts successful trajectories into procedures and organizes them in a procedure graph, where each transition is associated with a reliability record embedding to estimate task-specific contribution from past outcomes. Before each episode, the agent simulates future procedural trajectories over known graph structures as structured soft guidance; after execution, it refines the graph using environment feedback. The approach is evaluated on public benchmarks against strong baselines.

Key Results

Experiments on public benchmarks show that ProPlay consistently improves environment understanding and self-evolution capability over strong baselines.

Resources

Paper Code