World Models That Know When They Don't Know: Controllable Video Generation with Calibrated Uncertainty
TLDR
Proposes C3, an uncertainty quantification method for controllable video generation to estimate confidence and localize hallucination at subpatch level.
Reasoning
The paper addresses a critical problem of hallucination in video models by introducing a novel uncertainty quantification framework with three innovations. However, the abstract does not mention any real-world experiments, benchmarks, or empirical evaluations, limiting evidence of practical applicability.
Read-first score
Read-first score 58.6, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 41.
Field roles
Rank sensitivity
Stability: volatile; rank range: 198.
Keyword Scores
Deep Analysis
Innovations
- Training video models for correctness and calibration via strictly proper scoring rules
- Estimating uncertainty in latent space to avoid training instability and prohibitive costs of pixel-space approaches
- Mapping dense latent-space uncertainty to interpretable pixel-level uncertainty in RGB space for high-resolution uncertainty heatmaps
Methodology
C3 is an uncertainty quantification method for controllable video generation models. It trains models using strictly proper scoring rules to achieve calibration, estimates uncertainty in latent space to reduce training cost and instability, and then maps the latent uncertainty to pixel-level uncertainty in RGB space for interpretable heatmaps. The method is evaluated on large-scale robot learning datasets Bridge and DROID, along with real-world evaluations.
Key Results
The method provides calibrated uncertainty estimates within the training distribution and enables effective out-of-distribution detection, as demonstrated through extensive experiments on robot learning datasets and real-world evaluations.