AdaptiveLoad: Towards Efficient Video Diffusion Transformer Training

arXiv 2026 54 system

TLDR

AdaptiveLoad optimizes video diffusion transformer training for world models via dual-constraint load balancing and fused kernel, improving throughput by 27.2%.

Reasoning

The paper addresses a critical computational bottleneck in training video diffusion transformers for world models, proposing a novel load balancing system and fused CUDA kernel. Strengths include clear problem identification and significant empirical gains on a real-world model (Wan 2.1). Weaknesses are limited generality (single architecture) and lack of comparison to other advanced scheduling methods.

Read-first score

Read-first score 54, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 35.

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Citation impact 18%

81.2

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. citation_normalized_percentile=0.81155173

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=dataset,experiment,result

Topical relevance 29%

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=dataset

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FoundationFrontierBridgeMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 244.

Keyword Scores

world model

video world model

generative world model

world simulator

world dynamics prediction

interactive world model

model-based reinforcement learning world model

Deep Analysis

Innovations

Dual-constraint adaptive load balancing system that simultaneously limits memory consumption and computational load using B × S^p ≤ M_comp
Fused LayerNorm-Modulate CUDA kernel utilizing D-tile coalesced reduction strategy to increase throughput and alleviate memory pressure

Methodology

AdaptiveLoad is an integrated optimization framework for training video diffusion Transformers. It consists of two components: (1) a dual-constraint adaptive load balancing system that eliminates long-sequence bottlenecks by constraining both memory and computation (B × S^p ≤ M_comp), and (2) a fused LayerNorm-Modulate CUDA kernel using D-tile coalesced reduction. The method is evaluated on the Wan 2.1 world model.

Key Results

The method reduces computational imbalance rate from 39% to 18.9%, improves peak VRAM utilization efficiency by 22.7%, and achieves an overall training throughput increase of 27.2% on the Wan 2.1 world model.

Resources

Paper