Divide and Conquer: Decoupled Representation Alignment for Multimodal World Models

arXiv 2026 48.9 method

TLDR

Proposes M^2-REPA, a representation alignment method for multi-modal video generation that decouples and aligns modality-specific features with expert foundation models.

Reasoning

The paper introduces a novel decoupling and alignment approach leveraging multiple foundation models as experts, which is a strength. However, it focuses solely on video generation without addressing interactive or reinforcement learning aspects, limiting its scope.

Read-first score

Read-first score 48.9, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 35.

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Citation impact 18%

70.7

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. citation_normalized_percentile=0.70711964

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,experiment

Topical relevance 29%

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FrontierBridge

Rank sensitivity

Stability: volatile; rank range: 240.

Keyword Scores

world model

video world model

generative world model

world dynamics prediction

world simulator

interactive world model

model-based reinforcement learning world model

Deep Analysis

Innovations

First representation alignment method tailored for multi-modal video generation (M^2-REPA)
Decoupling modality-specific features from diffusion model's intermediate representations and aligning each with corresponding expert foundation models
Two synergistic objectives: multi-modal representation alignment loss for feature-to-expert matching and modality-specific decoupling regularization for complementarity

Methodology

The method decouples modality-specific features from a diffusion model's intermediate representations and aligns each with a corresponding expert foundation model (e.g., RGB, depth, mask). It uses two loss functions: a multi-modal representation alignment loss for feature-to-expert matching and a modality-specific decoupling regularization to encourage complementarity across modalities. The model is trained jointly to exploit priors from multiple foundation models.

Key Results

The method significantly outperforms baselines in visual quality and long-term consistency.

Resources

Paper