Latent-Foresight: End-to-End Learning Predictable Representations for Latent World Models

arXiv 2026 37.9 method

TLDR

Proposes Latent-Foresight, an end-to-end framework jointly learning a latent tokenizer and flow-based generative dynamics model to create predictable latent representations for world modeling.

Reasoning

The paper addresses a clear limitation of two-stage world models by jointly optimizing representation learning and dynamics prediction, with design choices to prevent latent collapse. Strong empirical claims are made across future scene understanding tasks, but the abstract lacks specific benchmark details and does not address interactive or RL settings.

Read-first score

Read-first score 37.9, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 41.

Recency 5%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Methodology quality 15%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,experiment,result

Topical relevance 25%

58.6

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 15%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=code,github

Graph prestige 15%

5.7

PageRank-style prestige over the local citation graph built from available OpenAlex references. local_pagerank=0.0016

Citation impact 15%

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. cited_by_count=0

Citation velocity 10%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 488.

Keyword Scores

world model

world dynamics prediction

generative world model

video world model

world simulator

interactive world model

model-based reinforcement learning world model

Deep Analysis

Innovations

End-to-end joint learning of a latent tokenizer and a flow-based generative dynamics model, rather than using two-stage pipelines with frozen compressed features.
Representation learning explicitly shaped for temporal predictability, addressing the lack of predictability guarantees in fixed dimensionality reduction, independently trained autoencoders, or raw VFM features.
Key design choices that prevent latent collapse and align reconstruction with generative objectives during joint optimization.
Elimination of separate training stages, including during high-resolution adaptation.

Methodology

Latent-Foresight jointly trains a latent tokenizer and a flow-based generative dynamics model, with design choices to avoid latent collapse and align reconstruction with generative objectives. The approach is evaluated on multiple future scene understanding tasks and prediction horizons against two-stage baselines, including high-resolution adaptation.

Key Results

Latent-Foresight learns more temporally coherent latent representations and consistently outperforms two-stage baselines across multiple future scene understanding tasks and prediction horizons, while removing separate training stages.

Resources

Paper