Masked Visual Actions for Unified World Modeling

arXiv 2026 48.3 method, application

TLDR

Introduces Masked Visual Actions, a pixel-space interface for action communication to video models, enabling unified world modeling for robotics.

Reasoning

The paper's strength lies in its novel action representation that aligns with visual priors, requiring only 15 hours of fine-tuning. Weaknesses include potential limitations to visible manipulation tasks and lack of explicit comparison to other world model methods.

Read-first score

Read-first score 48.3, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 64.

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Topical relevance 29%

91.4

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=evaluation

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=checkpoint

Citation impact 18%

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. cited_by_count=0

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

Frontier

Rank sensitivity

Stability: volatile; rank range: 648.

Keyword Scores

world model

video world model

world simulator

generative world model

interactive world model

world dynamics prediction

model-based reinforcement learning world model

Deep Analysis

Innovations

Masked Visual Actions: a pixel-space control interface that represents action as a partially revealed trajectory of an arbitrary entity in a video, enabling unified forward and inverse dynamics modeling.
Unified forward and inverse dynamics: revealing robot motion makes the model act as a forward dynamics predictor, while revealing desired object motion makes it recover robot behavior consistent with that outcome, all in a single model.
Efficient fine-tuning: achieves strong visual fidelity and controllability using only 15 hours of masked examples from real videos and simulation.
Cross-embodiment controllability: a single checkpoint generalizes across diverse scenes and multiple embodiments.

Methodology

The approach fine-tunes a video model with Masked Visual Actions, where actions are pixel-space masks revealing part of a future trajectory (robot or object). Revealing robot motion conditions the model to predict the scene's response as a forward dynamics model; revealing desired object motion conditions it to generate the robot motion achieving that outcome. Training uses 15 hours of masked examples from real-world videos and simulation.

Key Results

The model generates imagined rollouts whose outcomes correlate with real-world execution, improves decision-making in model-based planning by ranking candidate futures, and synthesizes robot motion from desired object motion, all with a single checkpoint.

Resources

Paper