Can In-Context Learning Support Intrinsic Curiosity?

arXiv 2026 46.4 method

TLDR

The paper theoretically investigates using in-context learning as an update-free world model for intrinsic curiosity, proving impossibility in general MDPs but possibility in non-temporal settings.

Reasoning

Strengths: Clear theoretical contributions with both negative and positive results, addressing a practical bottleneck in exploration. Weaknesses: No empirical validation or real-world experiments; the abstract is incomplete (cut off), limiting full assessment.

Read-first score

Read-first score 46.4, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 14.

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Citation impact 18%

96.2

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. citation_normalized_percentile=0.96231625

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=experiment,result

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Topical relevance 29%

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FoundationFrontierBridgeMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 381.

Keyword Scores

world model

model-based reinforcement learning world model

world dynamics prediction

world simulator

generative world model

interactive world model

video world model

Deep Analysis

Innovations

Investigating whether in-context learning (ICL) can serve as update-free world models for intrinsic curiosity, eliminating expensive gradient inner loops.
Proving that in general Markov decision processes, unbiased estimation of learning progress using ICL prediction errors is impossible due to nuisance terms or implementation constraints.
Proving a positive result for non-temporal settings (active learning, Bayesian Experimental Design) where ICL-derived rewards bound and asymptotically converge to true learning progress.
Proposing a framework that trains curious data-collection policies using ICL prediction errors and counterfactual context manipulations.

Methodology

The study combines theoretical proofs (impossibility for general MDPs, positive results for non-temporal settings) with controlled experiments in continuous and symbolic environments. An exploration policy is trained to maximize learning progress using only the prediction errors and counterfactual context manipulations of an in-context learner, without gradient updates within trajectories.

Key Results

The ICL-driven framework successfully trains curious data-collection policies that explore optimally in non-temporal settings, while general MDPs are shown to be impossible for unbiased estimation of learning progress via ICL.

Limitations

In general Markov decision processes, unbiased estimation of learning progress using ICL is impossible; the resulting intrinsic rewards either suffer from nuisance terms or cannot be implemented.
The positive result is limited to a broad subclass of non-temporal settings (active learning, Bayesian Experimental Design), not general sequential decision-making.
The approach relies on the emergent in-context learning capabilities of sequence models, which may have inherent limitations not explored in this work.
Experiments are conducted only in controlled continuous and symbolic environments, leaving real-world applicability unverified.

Resources

Paper