HappyWorld-Bench
TLDR
Introduces HappyWorld-Bench, a benchmark with three tracks and 1,138 video prompts, 300 spatial scenes, and 254 embodied cases to evaluate world models' generation, interaction, and consistency.
Reasoning
The paper provides a comprehensive evaluation framework with hierarchical capabilities, multiple tracks, human A/B comparisons, and automated metrics across many models. Weaknesses are not visible from abstract alone, but the benchmark's reliance on human comparisons may limit scalability.
Read-first score
Read-first score 44.6, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 58.
Field roles
Rank sensitivity
Stability: volatile; rank range: 681.
Keyword Scores
Deep Analysis
Innovations
- Introduces HappyWorld-Bench, a comprehensive benchmark for evaluating whether generated worlds remain reliable under exploration, interaction, and modification.
- Proposes a hierarchical capability framework of six world capabilities (W1-W6), from generative construction to unified world modeling.
- Instantiates the framework across three independent evaluation tracks: video world models, spatial world models, and embodied world models.
- Builds and operates HappyWorld-Arena to organize human A/B comparisons and derive model-level Elo ratings, complemented by newly designed automated metrics for behavioral correctness.
Methodology
HappyWorld-Bench comprises 1,138 video prompts, 300 spatial scenes, and 254 embodied test cases organized under the W1-W6 hierarchical framework. It evaluates 14 video world models, 9 spatial systems, and 8 embodied candidates across three tracks. Human A/B comparisons in HappyWorld-Arena produce Elo ratings, while newly designed automated metrics capture behavioral correctness.
Key Results
Video world models show reduced consistency during extended rollouts and revisits; spatial models achieve at best 70.14% placement accuracy and 73.33% edit execution; embodied models struggle to preserve state across multi-step actions and respond precisely to altered action conditions and physical rules.
Limitations
- No explicit limitations of the benchmark or study itself are stated in the abstract.