Beyond Successful Robot Demonstrations: Learning from Human Interactions and Modeling Failure Dynamics

arXiv 2026 43.6 method, application

TLDR

HF-WM, an action-conditioned world model, combines human interaction learning with failure-aware post-training to improve robot world modeling and failure dynamics prediction.

评分理由

The paper addresses important limitations in robot world models, namely cross-embodiment learning from human videos and underrepresented failure dynamics, with a novel dual-branch and GRPO-based approach. However, the abstract is truncated and lacks concrete experimental details, so the strength of the empirical claims cannot be fully assessed from the visible text.

Read-first 评分解释

综合优先阅读分 43.6，由主题、引用、图谱、方法、可复现性和近期性等信号加权得到。原始总分保留为 56。

近期性 5%

100

使用温和的时间衰减，让近期论文更容易浮现，同时保留较早基础工作的价值。年份：2026

主题相关性 25%

使用现有 LLM 关键词相关性评分，并归一化到 0-100。关键词：world model、world simulator、generative world model、interactive world model、video world model、world dynamics prediction、model-based reinforcement learning world model

方法质量 15%

检查可见的摘要与分析字段，寻找实验、数据集、基线、指标和局限性等方法证据。命中信号：数据集、评估、实验

可复现性 15%

检查链接和可见文本中的论文、代码、数据集、工件与仓库信号。论文：有；代码：无；数据：无；命中信号：数据集

图谱声望 15%

5.7

基于可用 OpenAlex 引用关系构建局部引用图，并计算 PageRank 风格的图谱声望。局部 PageRank：0.0016

引用影响力 15%

使用 OpenAlex 形态的引用元数据作为文献关注度信号，并与论文本身质量分开处理。引用数：0

引用速度 10%

引用速度按发表年限估算年均引用，降低旧论文天然占优的偏差。年均引用：0.00

研究版图角色

前沿论文方法锚点

排序敏感性

稳定性：volatile；排名波动范围：628。

关键词评分

world model

interactive world model

world dynamics prediction

generative world model

video world model

world simulator

model-based reinforcement learning world model

深度分析

创新点

Introduces HF-WM, an action-conditioned world model that combines human interaction learning with failure-aware post-training.
Proposes a dual-branch framework that couples video generation with auxiliary pointflow prediction, using stabilized pointflow as a unified motion representation across human and robot interactions while suppressing egocentric camera motion.
Develops a GRPO-based failure-aware post-training strategy that perturbs actions from successful demonstrations and uses action alignment and physical plausibility rewards to learn plausible failure dynamics without paired ground-truth videos.

方法

HF-WM is an action-conditioned world model that learns from both robotic and large-scale egocentric human video data. It uses a dual-branch framework coupling video generation with auxiliary pointflow prediction, where stabilized pointflow serves as a unified motion representation across embodiments and suppresses egocentric camera motion. The model is post-trained with Group Relative Policy Optimization on perturbed actions from successful demonstrations, guided by action alignment and physical plausibility rewards, and evaluated on action-conditioned video generation and downstream robotic policy evaluation.

关键结果

Extensive experiments demonstrate that HF-WM achieves state-of-the-art performance in action-conditioned video generation and shows strong utility for policy evaluation in downstream robotic tasks.

局限性

Existing robotic datasets provide limited coverage of the diverse interaction dynamics required for generalizable world modeling.
The embodiment gap between human hands and robotic manipulators hinders effective cross-embodiment learning.
Failure dynamics are substantially underrepresented because of the predominance of successful demonstrations.

资源

论文