PROWL: Prioritized Regret-Driven Optimization for World Model Learning

arXiv 2026 61.6 method

TLDR

PROWL uses adversarial curriculum and prioritized replay to improve robustness of video world models on rare, high-impact transitions.

Reasoning

The paper introduces a novel adversarial training loop for world models, combining a KL-constrained policy to expose failures and a prioritized buffer to focus on unresolved weaknesses. Strengths include a clear methodology and empirical evaluation in MineRL, but limitations include reliance on a specific simulation environment and potential sensitivity to behavioral constraints.

Read-first score

Read-first score 61.6, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 57.

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Citation impact 18%

81.9

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. citation_normalized_percentile=0.81948681

Topical relevance 29%

81.4

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=dataset,result

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=dataset

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FoundationFrontierBridge

Rank sensitivity

Stability: volatile; rank range: 481.

Keyword Scores

world model

video world model

world dynamics prediction

generative world model

interactive world model

model-based reinforcement learning world model

world simulator

Deep Analysis

Innovations

KL-constrained adversarial curriculum that trains a policy to expose high-error trajectories of a diffusion-based world model while staying close to the behavior distribution
Prioritized Adversarial Trajectory (PAT) buffer that re-ranks trajectories based on prediction error, action fidelity, and learning progress to focus training on unresolved failure modes

Methodology

The method introduces a KL-constrained adversarial curriculum where a policy is trained to discover high-error trajectories of a diffusion-based world model while remaining close to the behavior distribution. The world model is continuously fine-tuned on these adversarially discovered trajectories, forming an adversarial training loop. A Prioritized Adversarial Trajectory (PAT) buffer re-ranks trajectories using prediction error, action fidelity, and learning progress to maintain pressure on unresolved weaknesses.

Key Results

PROWL improves robustness over models trained on passive data alone on held-out out-of-distribution trajectories in the MineRL framework, reveals reward-hacking behaviors under weak behavioral constraints, and demonstrates that effective adversarial world-model training critically depends on balancing exploratory failure discovery with explicit behavioral regularization.

Resources

Paper