Self-Supervised Multi-Modal World Model with 4D Space-Time Embedding

World Modeling Workshop 26 2026 59.3 method

TLDR

DeepEarth introduces Earth4D, a 4D space-time positional encoder for self-supervised multi-modal world modeling, achieving SOTA on ecological forecasting.

Reasoning

Strengths include a novel 4D hash encoding that scales to planetary level, multi-modal fusion, and state-of-the-art results on a benchmark with open-source code. Weaknesses are limited evaluation to a single ecological forecasting task and no explicit demonstration of interactive or video capabilities.

Read-first score

Read-first score 59.3, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 29.

Recency 8%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=True; dataset=False; markers=code,github,open source

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark

Topical relevance 42%

41.4

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Field roles

FrontierReproducibility anchor

Rank sensitivity

Stability: volatile; rank range: 503.

Keyword Scores

world model

generative world model

world dynamics prediction

world simulator

video world model

interactive world model

model-based reinforcement learning world model

Deep Analysis

Innovations

Earth4D: a novel planetary-scale 4D space-time positional encoder extending 3D multi-resolution hash encoding to include time, enabling sub-meter, sub-second precision across centuries
Self-supervised multi-modal world model fusing vision-language encoders with Earth4D embeddings via masked reconstruction
Learnable hash probing that surpasses a multi-modal foundation model pre-trained on substantially more data

Methodology

DeepEarth uses Earth4D, a 4D space-time positional encoder that extends 3D multi-resolution hash encoding to incorporate time, allowing efficient planetary-scale coverage. Multi-modal encoders (e.g., vision-language models) are fused with Earth4D embeddings and the entire model is trained via masked reconstruction.

Key Results

Earth4D achieves state-of-the-art performance on an ecological forecasting benchmark. With learnable hash probing, it surpasses a multi-modal foundation model pre-trained on substantially more data.

Resources

Paper Project