RoboGaze: Evaluating Robot World Models via Structured Vision-Language Analysis

arXiv 2026 39.1 method

TLDR

RoboGaze is a training-free multi-agent VLM framework for structured, interpretable evaluation of robot-generated manipulation videos, with a taxonomy and human-validated dataset.

评分理由

The paper introduces a novel evaluation framework and dataset, showing strong gains over baselines and addressing false positives. Its main limitation is that it evaluates world models rather than proposing or improving them, and performance depends on underlying VLM backbones.

Read-first 评分解释

综合优先阅读分 39.1，由主题、引用、图谱、方法、可复现性和近期性等信号加权得到。原始总分保留为 35。

方法质量 15%

100

检查可见的摘要与分析字段，寻找实验、数据集、基线、指标和局限性等方法证据。命中信号：分析、基线、基准、数据集、评估、指标

近期性 5%

100

使用温和的时间衰减，让近期论文更容易浮现，同时保留较早基础工作的价值。年份：2026

主题相关性 25%

使用现有 LLM 关键词相关性评分，并归一化到 0-100。关键词：world model、world simulator、generative world model、interactive world model、video world model、world dynamics prediction、model-based reinforcement learning world model

可复现性 15%

检查链接和可见文本中的论文、代码、数据集、工件与仓库信号。论文：有；代码：无；数据：无；命中信号：数据集

图谱声望 15%

5.7

基于可用 OpenAlex 引用关系构建局部引用图，并计算 PageRank 风格的图谱声望。局部 PageRank：0.0016

引用影响力 15%

使用 OpenAlex 形态的引用元数据作为文献关注度信号，并与论文本身质量分开处理。引用数：0

引用速度 10%

引用速度按发表年限估算年均引用，降低旧论文天然占优的偏差。年均引用：0.00

研究版图角色

前沿论文方法锚点

排序敏感性

稳定性：volatile；排名波动范围：716。

关键词评分

world model

generative world model

video world model

interactive world model

world dynamics prediction

world simulator

model-based reinforcement learning world model

深度分析

创新点

Training-free multi-agent VLM framework (RoboGaze) for structured, interpretable evaluation of generated robot-manipulation videos.
Three-stage evaluation pipeline: task-scene grounding, dimension-specific specialist routing, and critic-based verification.
Novel 6-dimension, 30-type robotics-specific taxonomy producing temporally localized glitch reports.
Human-validated benchmark dataset of 382 clips spanning simulated and real-world multi-view manipulation.
Critic verifier designed to reduce false-positive 'cry-wolf' errors in standard VLM judges.

方法

RoboGaze is a training-free, multi-agent VLM framework that evaluates robot-manipulation videos from a task instruction and video. It uses a three-stage pipeline consisting of task-scene grounding, dimension-specific specialist routing, and critic-based verification to output temporally localized glitch reports under a 6-dimension, 30-type taxonomy. Evaluation uses a human-validated dataset of 382 simulated and real-world multi-view manipulation clips and compares eight open-source and proprietary VLM backbones against zero-shot baselines.

关键结果

RoboGaze improves description-F1 by up to +43 points and temporal alignment (F1 x IoU) by up to +37 points over zero-shot baselines, closing about 85% of the gap to the human ceiling. Its critic verifier raises clean-clip accuracy from under 25% to over 80%.

资源

论文