⌁ Awesome World Model Hub Papers · Datasets · Projects
中文 GitHub
← Back to papers

Generating Videos with Scene Dynamics

arXiv 2016 46.6 method

TLDR

Proposes a generative adversarial network with spatio-temporal convolutions to learn scene dynamics from unlabeled video, enabling future prediction and action recognition.

Reasoning

The paper introduces a novel video GAN architecture that separates foreground and background to model scene dynamics, with experiments on future prediction and action recognition. Weaknesses include limited evaluation on tiny videos and simple baselines, and no interactive or reinforcement learning components.

Read-first score

Read-first score 46.6, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 33.

Methodology quality 25%
70

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,experiment

Topical relevance 42%
47.1

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 25%
30

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Recency 8%
24

Uses a gentle age decay so recent papers surface without erasing older foundations. 2016

Field roles

Methodology anchor

Rank sensitivity

Stability: volatile; rank range: 311.

Keyword Scores

video world model
8
world dynamics prediction
8
generative world model
7
world model
6
world simulator
4
interactive world model
0
model-based reinforcement learning world model
0

Deep Analysis

Innovations

  • Proposes a generative adversarial network for video with a spatio-temporal convolutional architecture that separates foreground from background.
  • Leverages large amounts of unlabeled video to learn scene dynamics for both video generation and recognition, using scene dynamics as a signal for representation learning.

Methodology

A generative adversarial network with a spatio-temporal convolutional architecture that untangles foreground and background is trained on large-scale unlabeled video. The model is evaluated on video generation (future prediction from static images) and action recognition with minimal supervision.

Key Results

The model generates tiny videos up to a second at full frame rate better than simple baselines, produces plausible futures from static images, and its learned features are useful for action recognition with minimal supervision.

Limitations

  • Generated videos are limited to tiny size and duration up to one second.
  • Performance is only compared against simple baselines, not more advanced methods.

Resources

Paper

Tags