Turning Video Models into Generalist Robot Policies

arXiv 2026 55.7 method, application

TLDR

Decoupling video generative models as planners from embodiment-specific inverse dynamics models enables generalist robot policies with strong real-world performance.

Reasoning

The paper presents a novel decoupled approach (VERA) that separates video planning from action prediction, demonstrating strong empirical results across simulated and real-world benchmarks. Its strengths include data efficiency, embodiment-agnostic video models, and zero-shot transfer; however, the abstract does not discuss limitations or comparisons to end-to-end methods.

Read-first score

Read-first score 55.7, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 40.

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark,evaluation,result

Citation impact 18%

77.4

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. citation_normalized_percentile=0.77401095

Topical relevance 29%

57.1

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FoundationFrontierBridgeMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 256.

Keyword Scores

video world model

world model

generative world model

world dynamics prediction

model-based reinforcement learning world model

world simulator

interactive world model

Deep Analysis

Innovations

Decoupling video planning from action prediction by using an inverse dynamics model (IDM) instead of finetuning the video model with action labels
IDM design based on the robot embodiment Jacobian for data efficiency and scalability to high-dimensional action spaces
VERA system enabling zero-shot cross-embodiment robot control using a single action-free video world model paired with embodiment-specific IDMs

Methodology

The paper proposes VERA, a closed-loop video-to-action policy that combines an action-free video world model (pretrained video generative model) with a carefully-designed inverse dynamics model (IDM) based on the robot embodiment Jacobian. The IDM is trained separately using readily available self-play data, while the video planner remains unchanged and embodiment-agnostic. Evaluation is performed on simulated and real-world benchmarks including zero-shot Panda arm manipulation and 16-DoF Allegro-hand dexterous cube re-orientation.

Key Results

VERA achieves strong performance across simulated and real-world benchmarks, including zero-shot Panda arm manipulation and 16-DoF Allegro-hand dexterous cube re-orientation. The same video planner can be used across multiple embodiments by pairing it with different embodiment-specific IDMs.

Limitations

Requires training an embodiment-specific IDM for each new robot, limiting full generality
Performance depends on the quality and accuracy of the action-free video world model for generating future observations
The IDM design based on the robot embodiment Jacobian may not generalize to all robot morphologies or action spaces

Resources

Paper