Ego2Act: Evaluating Goal-Directed Manipulation in Egocentric Video Generation

arXiv 2026 37.4 benchmark

TLDR

Ego2Act is a benchmark with 2,640 egocentric videos across 110 real-world tasks evaluating video generation models' ability to simulate goal-directed multi-step manipulation.

Reasoning

The paper addresses an important gap by evaluating video generation models on multi-step, goal-directed egocentric manipulation using real-world videos and a reference-free judge. Its main weakness is that the abstract is truncated before fully reporting quantitative results or limitations, and the contribution is primarily evaluative rather than a new generative method.

Read-first score

Read-first score 37.4, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 42.

Recency 5%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Methodology quality 15%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,benchmark,evaluation

Topical relevance 25%

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 15%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Graph prestige 15%

5.7

PageRank-style prestige over the local citation graph built from available OpenAlex references. local_pagerank=0.0016

Citation impact 15%

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. cited_by_count=0

Citation velocity 10%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 418.

Keyword Scores

world simulator

video world model

world model

generative world model

world dynamics prediction

interactive world model

model-based reinforcement learning world model

Deep Analysis

Innovations

Introduces Ego2Act, a goal-directed egocentric video generation benchmark with 2,640 videos from 110 real-world tasks across day-to-day settings, varying object clutter and multi-step complexity.
Introduces Ego2ActJudge, a reference-free evaluation pipeline for task completion and physics plausibility that aligns better with human consensus than relevant baselines.

Methodology

Ego2Act evaluates video generation models by providing an initial scene image and a high-level goal, then assessing whether the generated egocentric video shows a hand manipulating objects to complete the task. The benchmark includes 2,640 videos covering 110 real-world tasks with varied clutter and multi-step complexity, and Ego2ActJudge provides reference-free evaluation of task completion and physics plausibility against human consensus.

Key Results

Ego2ActJudge achieves better alignment with human consensus on task completion and physics plausibility than relevant baselines. Evaluated models often skip or only partially execute steps, leaving later steps missing dependent states and goals unfulfilled, and they consistently fail at fine-grained physical dynamics in complex object manipulation and persistent world modeling.

Limitations

Current video generation models often skip or partially execute steps, leaving later steps without required dependent states and unfulfilled goals.
Models consistently fail at fine-grained physical dynamics, particularly during complex object manipulation and persistent world modeling.

Resources

Paper