ShadowDancer: Teaching Video World Models Any Action by Learning Unified Dynamics Representations from a Video and Its Shadow

arXiv 2026 44 method

TLDR

ShadowDancer uses shadow pairs and cross-shadow prediction to enable any-action control of video world models from demonstrations without action labels.

Reasoning

The paper presents a novel approach to action representation in interactive video world models, leveraging shadow pairs to disentangle dynamics from appearance. Strengths include the ability to learn actions from demonstrations without labels or fine-tuning, enabling transfer across scenes. Weaknesses: the abstract is cut off, leaving experimental details incomplete, and the method's scalability to diverse dynamics is not fully validated.

Read-first score

Read-first score 44, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 54.

Recency 5%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Methodology quality 15%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,experiment,result

Topical relevance 25%

77.1

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 15%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=code,github

Graph prestige 15%

5.7

PageRank-style prestige over the local citation graph built from available OpenAlex references. local_pagerank=0.0016

Citation impact 15%

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. cited_by_count=0

Citation velocity 10%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 641.

Keyword Scores

video world model

world model

interactive world model

world dynamics prediction

generative world model

world simulator

model-based reinforcement learning world model

Deep Analysis

Innovations

Shadow pairs: video pairs replaying the same dynamics under independently resampled appearance, constructed at scale by a Shadow Library
Cross-shadow prediction: learning a unified dynamics representation by predicting one shadow from the other, discarding resampled appearance and preserving the action, driving a block-causal world model

Methodology

ShadowDancer constructs shadow pairs—video pairs with identical dynamics but different appearance—using a Shadow Library. It then learns a unified dynamics representation via cross-shadow prediction, where one shadow is predicted from the other, forcing the model to discard appearance and capture the underlying action. This representation drives a block-causal world model, enabling any demonstration clip to be reused as a frame-level action in new environments without action labels, motion estimators, or fine-tuning.

Key Results

ShadowDancer achieves improved action transfer and long action rollout over strong latent-action and interactive world model baselines across diverse dynamics families, with an average blinded win rate of 86% in rollout comparisons.

Limitations

A dynamics family becomes controllable only when shadow pairs can be constructed for it, limiting applicability to domains where such paired data can be generated.

Resources

Paper