Sekai2: From World Exploration to Interactive World Modeling

arXiv 2026 38.8 benchmark

TLDR

Introduces Sekai2, a large real-world video dataset with camera trajectories, hierarchical annotations, and panoramic sequences to support interactive video world modeling.

Reasoning

The paper presents a substantial dataset with long videos, camera trajectories, semantics, and looped panoramic observations, which is highly relevant for world model research. However, it lacks downstream experiments or model evaluations, relying only on dataset construction and statistics.

Read-first score

Read-first score 38.8, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 47.

Recency 5%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Methodology quality 15%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=dataset,metric,result

Topical relevance 25%

67.1

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 15%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=dataset

Graph prestige 15%

5.7

PageRank-style prestige over the local citation graph built from available OpenAlex references. local_pagerank=0.0016

Citation impact 15%

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. cited_by_count=0

Citation velocity 10%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 423.

Keyword Scores

world model

interactive world model

video world model

generative world model

world dynamics prediction

world simulator

model-based reinforcement learning world model

Deep Analysis

Innovations

Introduces Sekai2, a multi-source real-world video dataset designed for interactive world modeling, with camera trajectories and hierarchical temporally grounded semantic annotations for every clip.
Includes 982 panoramic sequences captured along non-linear trajectories with loops and revisits, providing repeated observations of the same locations across time and viewpoints to support persistent scene representations, long-term spatial memory, and geometrically consistent world models.
Provides hierarchical annotations that disentangle subject motion, environment dynamics, static scene content, and camera behavior, yielding 649,597 temporally grounded segments.
Deliberately weights the dataset toward sustained observation, with 43,594 segments reaching a full two minutes under a common 120-second decomposition, accounting for 51.4% of all footage.

Methodology

Sekai2 aggregates 10,428 source videos from 113 countries or regions into 128,892 clips totaling 2,826 hours, with every clip including a released camera trajectory and hierarchical annotations across subject motion, environment dynamics, static scene content, and camera behavior. It further introduces 982 panoramic sequences shot along non-linear trajectories with loops and revisits to provide repeated same-location observations. Corpus-scale analyses evaluate pose-and-caption coverage, geographic and semantic diversity, camera trajectory variability, and temporal description non-redundancy.

Key Results

The dataset contains 128,892 clips totaling 2,826 hours from 10,428 source videos across 113 countries or regions, with 43,594 full 120-second segments representing 51.4% of footage, 649,597 temporally grounded segments, and 982 panoramic loop/revisit sequences. Corpus-scale analyses demonstrate complete pose-and-caption coverage, broad geographic and semantic diversity, varied camera trajectories, and highly non-redundant temporal descriptions.

Resources

Paper