RoboJudge: Multimodal Language Models as Judges for Embodied Video Generation

arXiv 2026 31 method

TLDR

Introduces RoboJudge-Bench to evaluate multimodal LLMs as judges of embodied video generation, plus RoboJudge-Train and fine-tuned RoboJudge-9B achieving highest human agreement.

Reasoning

The paper introduces a well-structured benchmark with human annotations and diverse video sources, addressing an important evaluation gap. Its main limitation is that it evaluates judges rather than proposing or improving video world models themselves, so relevance to world-model keywords is indirect.

Read-first score

Read-first score 31, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 25.

Recency 5%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Methodology quality 15%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,benchmark,dataset,evaluation,experiment

Reproducibility 15%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=dataset

Topical relevance 25%

35.7

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Graph prestige 15%

5.7

PageRank-style prestige over the local citation graph built from available OpenAlex references. local_pagerank=0.0016

Citation impact 15%

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. cited_by_count=0

Citation velocity 10%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 276.

Keyword Scores

video world model

world model

world simulator

generative world model

world dynamics prediction

interactive world model

model-based reinforcement learning world model

Resources

Paper