World Properties without World Models: Recovering Spatial and Temporal Structure from Co-occurrence Statistics in Static Word Embeddings

arXiv 26.3 2026 31.4 method

TLDR

Static word embeddings recover spatial and temporal structure from co-occurrence, challenging claims that such recovery implies world models.

Reasoning

The paper's strength is demonstrating that simple co-occurrence-based embeddings (GloVe, Word2Vec) can recover geographic and temporal variables, suggesting that linear probe recoverability does not require world models. A weakness is that it only tests linear probes and does not address more complex world model architectures or interactive settings.

Read-first score

Read-first score 31.4, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 1.

Recency 8%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=ablation

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Topical relevance 42%

1.4

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Field roles

Frontier

Rank sensitivity

Stability: volatile; rank range: 261.

Keyword Scores

world model

world simulator

generative world model

interactive world model

video world model

world dynamics prediction

model-based reinforcement learning world model

Deep Analysis

Innovations

Demonstrating that geographic and temporal structure can be recovered from static co-occurrence embeddings (GloVe, Word2Vec) using linear probes, challenging the interpretation that such recoverability in LLMs implies world models.
Identifying that the recoverable signals depend on interpretable lexical gradients (country names, climate vocabulary) rather than abstract world representations.
Showing that simple static embeddings preserve richer spatial, temporal, and environmental structure than commonly assumed.

Methodology

The authors use ridge regression probes on static word embeddings (GloVe and Word2Vec) to predict city coordinates and historical birth years. They also perform semantic-neighbor analyses and targeted subspace ablations to understand the source of the signals.

Key Results

Held-out R^2 values of 0.71-0.87 for city coordinates and 0.48-0.52 for historical birth years, indicating substantial geographic signal and weaker but reliable temporal signal.

Limitations

Only tested on static co-occurrence embeddings (GloVe, Word2Vec), not on other embedding types or LLMs directly.
Temporal signal is weaker than geographic signal (R^2 0.48-0.52 vs 0.71-0.87), suggesting limited recoverability for temporal variables.
Recoverable signal depends heavily on interpretable lexical gradients (e.g., country names, climate vocabulary), which may reflect surface correlations rather than genuine world structure.
The study does not directly compare with LLM hidden states to quantify the additional structure beyond co-occurrence statistics.

Resources

Paper