One-shot World Models Using a Transformer Trained on a Synthetic Prior

arXiv 24.9 2024 54.8 method

TLDR

Proposes One-Shot World Model (OSWM), a transformer trained on synthetic data via in-context learning, adapting to simple environments with 1k context steps.

Reasoning

Strengths: novel use of synthetic prior and in-context learning for world models, enabling quick adaptation. Weaknesses: limited to simple environments, no real-world validation, and transfer to complex environments remains challenging.

Read-first score

Read-first score 54.8, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 48.

Recency 8%

75.1

Uses a gentle age decay so recent papers surface without erasing older foundations. 2024

Topical relevance 42%

68.6

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=none

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Field roles

Candidate

Rank sensitivity

Stability: volatile; rank range: 434.

Keyword Scores

world model

world dynamics prediction

model-based reinforcement learning world model

world simulator

interactive world model

generative world model

video world model

Deep Analysis

Innovations

Proposes One-Shot World Model (OSWM), a transformer world model learned purely from synthetic data via in-context learning, without requiring real environment observations.
Uses a prior composed of multiple randomly initialized neural networks to model dynamics of each state and reward dimension, enabling generalization to new environments.
Adopts supervised learning procedure of Prior-Fitted Networks with masking to enable probabilistic predictions from limited context.

Methodology

OSWM is a transformer trained in an in-context learning fashion on synthetic data sampled from a prior distribution of randomly initialized neural networks. The training procedure masks next-state and reward at random context positions and queries the model to make probabilistic predictions based on the remaining transition context, following the Prior-Fitted Networks approach.

Key Results

With only 1,000 transition steps as context, OSWM quickly adapts to simple grid world, CartPole gym, and a custom control environment, and successfully trains environment-solving agent policies.

Limitations

Transferring to more complex environments remains a challenge.
The approach is currently limited to relatively simple environments and may not scale to high-dimensional or complex dynamics.

Resources

Paper