Latent Policy Steering with Embodiment-Agnostic Pretrained World Models

arXiv 25.7 2025 63.8 method, application

TLDR

Pretraining world models on multi-embodiment data with optical flow improves robot visuomotor policies in low-data regimes via latent policy steering.

Reasoning

The paper presents a novel method (LPS) that leverages embodiment-agnostic world models pretrained on diverse data, showing clear improvements in both simulated and real-world tasks. Strengths include real-world validation and a practical approach to data scarcity; weaknesses include reliance on optical flow and limited analysis of failure cases.

Read-first score

Read-first score 63.8, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 38.

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,dataset,experiment,result

Recency 8%

86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Topical relevance 42%

54.3

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=dataset,github

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 254.

Keyword Scores

world model

world dynamics prediction

model-based reinforcement learning world model

generative world model

video world model

interactive world model

world simulator

Deep Analysis

Innovations

Using optical flow as an embodiment-agnostic action representation to pretrain world models on multi-embodiment data (robots and humans), enabling leverage of sub-optimal data.
Latent Policy Steering (LPS): a method that finetunes a pretrained world model on a small set of target-embodiment demonstrations, then uses the finetuned world model and a learned value function to evaluate and select the best action candidates from a base policy.

Methodology

The approach first pretrains a world model using optical flow as an embodiment-agnostic action representation on data from multiple embodiments (robots, humans). Given a small set of demonstrations on a target embodiment, the world model is finetuned to better align predictions, and a base policy and robust value function are trained. During inference, the finetuned world model and value function evaluate action candidates from the base policy and select the best one to improve performance.

Key Results

LPS improves behavior-cloned policies by 10.6% on average across four Robomimic tasks. In real-world experiments, LPS achieves a 70% relative improvement with 30-50 target-embodiment demonstrations and a 44% relative improvement with 60-100 demonstrations compared to a behavior-cloned baseline.

Limitations

The method relies on optical flow as an embodiment-agnostic action representation, which may not capture all action modalities (e.g., force, torque) or fine-grained manipulation details.
The approach requires finetuning on target-embodiment demonstrations, adding an extra training step compared to direct policy learning.
Performance is evaluated only in low-data regimes; scalability to large datasets or different task domains is not addressed.

Resources

Paper