RynnWorld-Teleop: An Action-Conditioned World Model for Digital Teleoperation

arXiv 2026 40.7 method, system

TLDR

Introduces digital teleoperation using a generative world model to synthesize egocentric videos from hand poses, enabling scalable data collection for imitation learning.

Reasoning

The paper presents a novel paradigm that decouples teleoperation from physical hardware, with strong empirical results including real-time generation (40+ FPS) and zero-shot Sim2Real transfer. However, the abstract lacks detailed comparisons to existing world models and does not discuss limitations such as generalization across diverse environments.

Read-first score

Read-first score 40.7, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 46.

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Topical relevance 29%

65.7

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=dataset

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=dataset

Citation impact 18%

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. cited_by_count=0

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

Frontier

Rank sensitivity

Stability: volatile; rank range: 292.

Keyword Scores

world model

generative world model

video world model

world dynamics prediction

world simulator

interactive world model

model-based reinforcement learning world model

Deep Analysis

Innovations

Digital teleoperation paradigm: decouples data collection from physical constraints by using a generative world model to synthesize egocentric videos from hand-pose streams.
Action-conditioned world model with depth-aware skeletal conditioning, progressive human-to-robot training on a video Diffusion Transformer, and streaming autoregressive distillation enabling real-time (40+ FPS) generation.
Policies trained solely on synthetic data generated by RynnWorld-Teleop achieve zero-shot Sim2Real transfer for dexterous bimanual tasks.
Using digitally teleoperated data to augment real-world datasets improves success rates of robotic policies.

Methodology

RynnWorld-Teleop employs a video Diffusion Transformer conditioned on an operator's hand-pose stream and a single reference image, incorporating depth-aware skeletal conditioning and progressive human-to-robot training. The model is distilled via streaming autoregressive distillation into a single-pass inference, achieving real-time interactive generation at 40+ FPS on a single H100 GPU. The generated pose stream serves as an embodiment-agnostic action label, enabling policy training for imitation learning.

Key Results

Policies trained exclusively on RynnWorld-Teleop-generated data achieve effective zero-shot Sim2Real transfer on dexterous bimanual tasks, and augmenting real-world datasets with the generated data consistently improves success rates.

Resources

Paper