Targeting World Models to Compromise Robot Learning Pipelines
TLDR
World models introduce a stealthy data poisoning attack vector in robot learning, enabling compromised policies despite safe training data.
Reasoning
The paper identifies a novel security vulnerability in world models used for robot learning, demonstrating effective attacks against state-of-the-art models. Strengths include a clear attack methodology and empirical validation; weaknesses include limited scope to specific attack types and potential lack of generality across all world model variants.
Read-first score
Read-first score 59.4, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 37.
Field roles
Rank sensitivity
Stability: volatile; rank range: 430.
Keyword Scores
Deep Analysis
Innovations
- Novel attack methods that inject malicious prompts or compromising transition dynamics into visibly safe teleoperated datasets, which are only activated when fed through a world model as input.
- Demonstration of a full end-to-end backdoor on a downstream DRL policy and a proof-of-concept for the VLA setting using state-of-the-art action-conditioned and text-conditioned world models.
Methodology
The authors propose data poisoning attacks that target world models by embedding malicious prompts or transition dynamics into teleoperated datasets that appear safe. These poisoned datasets are then used as input to world models, which generate synthetic dangerous robot training trajectories. The attacks are evaluated against both action-conditioned and text-conditioned world models, with downstream effects measured on a DRL policy (full backdoor) and a VLA setting (proof-of-concept).
Key Results
The attacks successfully produce synthetic dangerous robot training trajectories, leading to unsafe or compromised robot policies. A full end-to-end backdoor is demonstrated on a downstream DRL policy, and a proof-of-concept is shown for the VLA setting.
Limitations
- World models are vulnerable to stealthy data poisoning attacks that are difficult to detect using traditional methods.
- The findings highlight the need for more secure world models and a reevaluation of their role in the robot learning supply chain, implying current defenses are insufficient.