TraceGen: World Modeling in 3D Trace Space Enables Learning from Cross-Embodiment Videos

arXiv 25.11 2025 57.9 method

TLDR

TraceGen learns a world model in 3D trace-space from cross-embodiment videos, enabling few-shot robot task learning with fast inference.

Reasoning

Strengths include a novel symbolic representation, large-scale pretraining, and strong few-shot real-world results. Weaknesses: limited detail on task diversity and potential limitations of trace-space abstraction.

Read-first score

Read-first score 57.9, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 39.

Recency 8%

86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=evaluation,metric,result

Topical relevance 42%

55.7

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 240.

Keyword Scores

world model

world dynamics prediction

generative world model

model-based reinforcement learning world model

world simulator

video world model

interactive world model

Deep Analysis

Innovations

Introduction of a compact 3D 'trace-space' representation that unifies scene-level trajectories across embodiments, environments, and tasks.
TraceGen, a world model that predicts future motion in trace-space instead of pixel space, abstracting appearance while preserving geometric structure for manipulation.
TraceForge, a data pipeline that transforms heterogeneous human and robot videos into consistent 3D traces, producing a large-scale corpus of 123K videos and 1.8M observation-trace-language triplets.

Methodology

TraceGen is a world model trained on a large corpus of cross-embodiment videos transformed into a unified 3D trace-space via the TraceForge pipeline. The model predicts future motion in this symbolic representation, abstracting away visual appearance while retaining geometric structure. Training uses 123K videos and 1.8M observation-trace-language triplets, and the resulting motion prior is adapted to target tasks with only five demonstrations.

Key Results

With five target robot videos, TraceGen achieves 80% success across four tasks and offers 50-600x faster inference than state-of-the-art video-based world models. With only five uncalibrated human demonstration videos from a handheld phone, it reaches 67.5% success on a real robot.

Limitations

Performance drops from 80% to 67.5% when using human videos, indicating a remaining embodiment gap.
The approach relies on accurate 3D trace extraction from videos, which may be sensitive to reconstruction quality.
Evaluation is limited to four tasks; generalization to more complex or diverse tasks is not demonstrated.
Requires five demonstrations per task, still a small-data setting but not zero-shot.
The trace-space representation may lose fine-grained appearance cues that could be useful for certain manipulation tasks.

Resources

Paper