Learning Visual Feature-Based World Models via Residual Latent Action

arXiv 2026 58.9 method

TLDR

Proposes Residual Latent Action (RLA) and RLA-WM, a visual feature-based world model using flow matching, outperforming prior methods on simulation and real-world tasks.

Reasoning

The paper introduces a novel latent action representation (RLA) and a flow-matching-based world model that predicts visual features, achieving strong performance on both simulation and real-world datasets while being faster than video diffusion. Strengths include efficiency and reduced hallucination; weaknesses include limited discussion of limitations and potential complexity of flow matching.

Read-first score

Read-first score 58.9, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 53.

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Topical relevance 29%

75.7

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Citation impact 18%

74.4

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. citation_normalized_percentile=0.74366489

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=dataset

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=code,dataset,github

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FrontierBridge

Rank sensitivity

Stability: volatile; rank range: 434.

Keyword Scores

world model

world dynamics prediction

generative world model

model-based reinforcement learning world model

interactive world model

world simulator

video world model

Deep Analysis

Innovations

Residual Latent Action (RLA) representation learned from DINO residuals, which is predictive, generalizable, and encodes temporal progression
RLA World Model (RLA-WM) that predicts RLA values via flow matching, outperforming state-of-the-art feature-based and video-diffusion world models
Two robot learning techniques: a world action model with RLA from actionless demonstration videos, and a visual RL framework trained entirely inside a world model from offline videos only

Methodology

The paper introduces Residual Latent Action (RLA) learned from DINO residuals. The RLA World Model (RLA-WM) predicts future RLA values using flow matching. The model is evaluated on simulation and real-world datasets, compared against feature-based and video-diffusion world models, and used to develop two robot learning methods: a world action model from actionless videos and a visual RL framework trained solely on offline videos.

Key Results

RLA-WM outperforms both state-of-the-art feature-based and video-diffusion world models on simulation and real-world datasets, while being orders of magnitude faster than video diffusion.

Resources

Paper