ShadowDancer: Teaching Video World Models Any Action by Learning Unified Dynamics Representations from a Video and Its Shadow

arXiv 2026 50.8 method

TLDR

ShadowDancer uses shadow pairs and cross-shadow prediction to enable any-action control of video world models from demonstrations without action labels.

Reasoning

The paper presents a novel approach to action representation in interactive video world models, leveraging shadow pairs to disentangle dynamics from appearance. Strengths include the ability to learn actions from demonstrations without labels or fine-tuning, enabling transfer across scenes. Weaknesses: the abstract is cut off, leaving experimental details incomplete, and the method's scalability to diverse dynamics is not fully validated.

Read-first score

Read-first score 50.8, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 54.

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,experiment,result

Topical relevance 29%

77.1

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=code,github

Citation impact 18%

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. cited_by_count=0

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 628.

Keyword Scores

video world model

world model

interactive world model

world dynamics prediction

generative world model

world simulator

model-based reinforcement learning world model

Deep Analysis

Innovations

Shadow pairs: video pairs replaying the same dynamics under independently resampled appearance, constructed at scale by a Shadow Library
Cross-shadow prediction: learning a unified dynamics representation by predicting one shadow from the other, discarding resampled appearance and preserving the action, driving a block-causal world model

Methodology

ShadowDancer constructs shadow pairs—video pairs with identical dynamics but different appearance—using a Shadow Library. It then learns a unified dynamics representation via cross-shadow prediction, where one shadow is predicted from the other, forcing the model to discard appearance and capture the underlying action. This representation drives a block-causal world model, enabling any demonstration clip to be reused as a frame-level action in new environments without action labels, motion estimators, or fine-tuning.

Key Results

ShadowDancer achieves improved action transfer and long action rollout over strong latent-action and interactive world model baselines across diverse dynamics families, with an average blinded win rate of 86% in rollout comparisons.

Limitations

A dynamics family becomes controllable only when shadow pairs can be constructed for it, limiting applicability to domains where such paired data can be generated.

Resources

Paper