Hallucination in World Models is Predictable and Preventable
TLDR
Hallucination in world models stems from low data coverage; detectable via three signals and preventable with coverage-aware sampling and curiosity-driven finetuning.
Reasoning
The paper introduces a large dataset (MMBench2) and identifies three hallucination modes, offering both detection signals and mitigation strategies. Strengths include empirical grounding and practical finetuning with few trajectories; weaknesses are the reliance on a single model size (350M) and limited detail on generalizability beyond the dataset.
Read-first score
Read-first score 61.4, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 55.
Field roles
Rank sensitivity
Stability: volatile; rank range: 503.
Keyword Scores
Deep Analysis
Innovations
- Identification of three distinct hallucination modes: perceptual, action-marginalized, and scene-diverging
- Development of three signals that accurately predict where the world model will fail
- Coverage-aware sampling technique to close coverage gaps at training time
- Hallucination predictors used as curiosity rewards for targeted data collection, enabling data-efficient finetuning with as few as 50 real environment trajectories
- Introduction of MMBench2, a 427-hour, 210-task dataset for visual world modeling with ground-truth actions, rewards, and live simulators
Methodology
The authors train a 350M-parameter world model on the MMBench2 dataset. They identify three hallucination modes and develop three predictive signals. To mitigate, they use coverage-aware sampling during training and hallucination predictors as curiosity rewards for online data collection, enabling finetuning on unseen environments with few trajectories.
Key Results
Hallucination in world models is predictable and preventable; the same data-centric signals used for detection can also guide mitigation, allowing adaptation to unseen environments with as few as 50 real trajectories.