Self-Improving World Modelling with Latent Actions

arXiv 26.2 2026 55.6 method

TLDR

SWIRL: a self-improving framework for world modelling from state-only sequences using latent actions, variational inference, and RL, achieving gains on multiple benchmarks.

Reasoning

The paper presents a novel self-improvement approach that avoids costly action-labelled data by treating actions as latent, with theoretical guarantees and strong empirical results across diverse environments. Weaknesses include reliance on RL optimization (GRPO) which may be sample-inefficient, and limited discussion of failure cases or scalability.

Read-first score

Read-first score 55.6, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 50.

Recency 8%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Topical relevance 42%

71.4

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=none

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Field roles

Frontier

Rank sensitivity

Stability: volatile; rank range: 589.

Keyword Scores

world model

world dynamics prediction

generative world model

interactive world model

world simulator

model-based reinforcement learning world model

video world model

Deep Analysis

Innovations

Treating actions as a latent variable in world modelling, enabling learning from state-only sequences without action labels.
Alternating between Forward World Modelling (FWM) and Inverse Dynamics Modelling (IDM) in a self-improvement loop.
Using Variational Information Maximisation to update FWM by maximizing conditional mutual information between latent actions and next states given prior states.
Using ELBO Maximisation to update IDM to explain observed transitions, effectively performing coordinate ascent.
Training both models with reinforcement learning (GRPO) using the opposite frozen model's log-probability as a reward signal.
Providing theoretical learnability guarantees for both updates.

Methodology

SWIRL is a self-improvement framework that learns world models from state-only sequences by treating actions as a latent variable. It iterates two phases: (1) Variational Information Maximisation updates the Forward World Model (FWM) to generate next states that maximize conditional mutual information with latent actions given prior states; (2) ELBO Maximisation updates the Inverse Dynamics Model (IDM) to explain observed transitions. Both models are trained with reinforcement learning (GRPO) using the opposite frozen model's log-probability as a reward signal.

Key Results

SWIRL achieves gains of 16% on AURORABench, 28% on ByteMorph, 16% on WorldPredictionBench, and 14% on StableToolBench across LLM and VLM environments including single-turn and multi-turn open-world visual dynamics and synthetic textual environments for physics, web, and tool calling.

Resources

Paper