What Has a Foundation Model Found? Using Inductive Bias to Probe for World Models

ICML 25 2025 48.6 method

TLDR

Proposes inductive bias probe to evaluate if foundation models learn world models; finds they often fail to generalize beyond task-specific heuristics.

Reasoning

Strengths: novel evaluation technique, clear experimental setup across domains. Weaknesses: limited to synthetic data, no real-world validation; results may not generalize to all foundation models.

Read-first score

Read-first score 48.6, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 20.

Recency 8%

86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=analysis,dataset,experiment

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=dataset

Topical relevance 42%

28.6

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 480.

Keyword Scores

world model

world dynamics prediction

generative world model

world simulator

interactive world model

video world model

model-based reinforcement learning world model

Deep Analysis

Innovations

Development of an inductive bias probe technique to evaluate whether foundation models capture deeper world models by measuring alignment between the model's inductive bias and a postulated world model.
Discovery that foundation models can achieve high performance on training tasks yet fail to develop inductive biases towards the underlying world model when adapted to new tasks.
Specific finding that models trained on orbital trajectories consistently fail to apply Newtonian mechanics when adapted to new physics tasks, instead relying on task-specific heuristics.

Methodology

The authors propose an inductive bias probe that evaluates foundation models by examining how they adapt to synthetic datasets generated from a postulated world model. The technique measures the alignment between the model's inductive bias and the world model. Experiments are conducted across multiple domains, with a detailed focus on orbital trajectories and physics tasks, using synthetic data and adaptation tasks to assess generalization.

Key Results

Foundation models excel at their original training tasks but fail to develop inductive biases towards the underlying world model when adapted to new tasks. In particular, models trained on orbital trajectories consistently fail to apply Newtonian mechanics when adapted to new physics tasks, instead exhibiting task-specific heuristics that do not generalize.

Limitations

The probe relies on synthetic datasets generated from postulated world models, which may not capture the complexity and noise of real-world data.
The findings are primarily demonstrated on orbital trajectories; the generalizability to other domains and world models is not fully established.

Resources

Paper