WM-VLM: Probing Internal World Models for Interleaved Visual-Textual Reasoning

arXiv 2026 34.4 method

TLDR

Introduces WM-VLM, adding a lightweight world model branch to VLMs to generate intermediate visual states, improving 2D/3D mental rotation reasoning by up to 39.25 points.

Reasoning

Strengths include a clear two-stage training procedure, programmatically verifiable spatial tasks, strong gains over supervised fine-tuning, and ablations showing dependence on generated visual states. Weaknesses are that evaluations use synthetic mental rotation tasks rather than real-world benchmarks, and the world model scope is limited to next-state generation for spatial reasoning, not broader dynamics.

Read-first score

Read-first score 34.4, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 38.

Recency 5%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Methodology quality 15%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=ablation,evaluation,result

Topical relevance 25%

54.3

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 15%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Graph prestige 15%

5.7

PageRank-style prestige over the local citation graph built from available OpenAlex references. local_pagerank=0.0016

Citation impact 15%

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. cited_by_count=0

Citation velocity 10%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 278.

Keyword Scores

world model

generative world model

world simulator

world dynamics prediction

interactive world model

video world model

model-based reinforcement learning world model

Deep Analysis

Innovations

Introduces WM-VLM, equipping a pretrained VLM with a lightweight world model branch for generating intermediate visual states.
Uses a two-stage training process: first learn to generate the next visual state, then learn to reason using that generated state.
Constructs programmatic spatial reasoning tasks with verifiable intermediate visual states to evaluate both generation quality and reliance on visual states.
Demonstrates that generated visual states improve spatial reasoning compared with a language-only supervised fine-tuned VLM.

Methodology

WM-VLM adds a lightweight world model branch to a pretrained VLM to generate intermediate visual states. The model is trained in two stages: first to generate the next visual state, then to use that state for reasoning. Evaluation uses programmatically constructed 2D and 3D mental rotation tasks with verifiable intermediate visual states, comparing against a supervised fine-tuned backbone and ablating the generated states.

Key Results

WM-VLM consistently outperforms the supervised fine-tuned backbone on 2D and 3D mental rotation tasks, with improvements up to 39.25 percentage points. Ablations show that removing or corrupting the generated visual states sharply reduces performance.

Resources

Paper