Beyond Successful Robot Demonstrations: Learning from Human Interactions and Modeling Failure Dynamics

arXiv 2026 43.6 method, application

TLDR

HF-WM, an action-conditioned world model, combines human interaction learning with failure-aware post-training to improve robot world modeling and failure dynamics prediction.

Reasoning

The paper addresses important limitations in robot world models, namely cross-embodiment learning from human videos and underrepresented failure dynamics, with a novel dual-branch and GRPO-based approach. However, the abstract is truncated and lacks concrete experimental details, so the strength of the empirical claims cannot be fully assessed from the visible text.

Read-first score

Read-first score 43.6, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 56.

Recency 5%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Topical relevance 25%

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Methodology quality 15%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=dataset,evaluation,experiment

Reproducibility 15%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=dataset

Graph prestige 15%

5.7

PageRank-style prestige over the local citation graph built from available OpenAlex references. local_pagerank=0.0016

Citation impact 15%

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. cited_by_count=0

Citation velocity 10%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 628.

Keyword Scores

world model

interactive world model

world dynamics prediction

generative world model

video world model

world simulator

model-based reinforcement learning world model

Deep Analysis

Innovations

Introduces HF-WM, an action-conditioned world model that combines human interaction learning with failure-aware post-training.
Proposes a dual-branch framework that couples video generation with auxiliary pointflow prediction, using stabilized pointflow as a unified motion representation across human and robot interactions while suppressing egocentric camera motion.
Develops a GRPO-based failure-aware post-training strategy that perturbs actions from successful demonstrations and uses action alignment and physical plausibility rewards to learn plausible failure dynamics without paired ground-truth videos.

Methodology

HF-WM is an action-conditioned world model that learns from both robotic and large-scale egocentric human video data. It uses a dual-branch framework coupling video generation with auxiliary pointflow prediction, where stabilized pointflow serves as a unified motion representation across embodiments and suppresses egocentric camera motion. The model is post-trained with Group Relative Policy Optimization on perturbed actions from successful demonstrations, guided by action alignment and physical plausibility rewards, and evaluated on action-conditioned video generation and downstream robotic policy evaluation.

Key Results

Extensive experiments demonstrate that HF-WM achieves state-of-the-art performance in action-conditioned video generation and shows strong utility for policy evaluation in downstream robotic tasks.

Limitations

Existing robotic datasets provide limited coverage of the diverse interaction dynamics required for generalizable world modeling.
The embodiment gap between human hands and robotic manipulators hinders effective cross-embodiment learning.
Failure dynamics are substantially underrepresented because of the predominance of successful demonstrations.

Resources

Paper