GeoStream: Toward Precise Camera Controlled Streaming Video Generation
TLDR
GeoStream enables precise metric-scale camera control in autoregressive streaming video generation using a self-refreshing 3D cache and on-policy distillation.
Reasoning
The paper introduces a novel on-policy distillation method to align training and inference distributions, addressing autoregressive drift and geometric feedback. However, it focuses narrowly on camera control for video generation rather than a comprehensive world model, and lacks explicit connections to reinforcement learning.
Read-first score
Read-first score 59.4, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 46.
Field roles
Rank sensitivity
Stability: volatile; rank range: 449.
Keyword Scores
Deep Analysis
Innovations
- Self-refreshing 3D cache that is periodically updated online from the model's own outputs, enabling geometric conditioning beyond the initial frame frustum.
- On-policy distillation where conditioning during training is rendered from the student's own generated frames, aligning train and inference distributions and mitigating autoregressive drift and geometric feedback loop.
- Explicit metric-scale camera control in autoregressive streaming video generation, addressing limitations of prior non-autoregressive methods with static 3D caches.
Methodology
GeoStream maintains a self-refreshing 3D cache: it estimates depth from the most recently generated frame, unprojects to 3D, and reprojects into the target view to produce point reprojections as geometric conditioning for subsequent synthesis. Training uses on-policy distillation where the conditioning is rendered from the student's own generated frames, ensuring the model trains against the exact error distribution encountered at inference.
Key Results
Quantitative and qualitative results show that the approach substantially improves camera controllability compared to prior methods.
Limitations
- Relies on accurate depth estimation from generated frames, which may introduce errors that propagate through the cache.
- Online cache updates may increase computational overhead during inference.
- The abstract does not explicitly discuss performance on extremely long sequences or out-of-distribution camera trajectories beyond those tested.