Long-Horizon Audio-Visual Generation for Persistent Stories and Interactive Worlds
TLDR
JoyAI-Echo-1.5 is a unified audio-visual generation system with long-video and world-model variants, achieving state-of-the-art results on WBench and SANA-WM-Bench.
评分理由
The paper presents a strong unified system with novel memory and geometry-aware conditioning, supported by benchmark results. However, the abstract lacks details on limitations and ablations, and the world-model claims rely on standard benchmarks rather than real-world interactive deployment.
Read-first 评分解释
综合优先阅读分 47.5，由主题、引用、图谱、方法、可复现性和近期性等信号加权得到。 原始总分保留为 41。
研究版图角色
排序敏感性
稳定性：volatile；排名波动范围：689。
关键词评分
深度分析
创新点
- Unified audio-visual generation system (JoyAI-Echo-1.5) with two purpose-built variants for long-form video and interactive world generation.
- Composable cross-shot memory that aggregates visual evidence across multiple prior shots and speaker cues derived from speech-filtered full-shot audio to preserve character appearance and voice identity.
- Geometry-aware conditioning pathway that converts heterogeneous navigation inputs into calibrated metric 6-DoF camera trajectories for controller-agnostic interaction.
- Causal few-step generation via progressive teacher forcing and short- and long-horizon Self-Gradient Forcing on self-generated rollouts.
方法
JoyAI-Echo-1.5 transforms a bidirectional audio-visual backbone into a causal few-step generator using progressive teacher forcing and short- and long-horizon Self-Gradient Forcing on self-generated rollouts. The long-video variant conditions on text, image, and composable cross-shot memory, while the world-model variant injects geometry-aware calibrated 6-DoF camera trajectories. Evaluation uses existing long-video baselines and benchmarks including WBench and SANA-WM-Bench, with metrics such as cross-shot consistency, visual quality, text alignment, speech fidelity, and average benchmark score.
关键结果
JoyAI-Echo-1.5 improves over existing long-video baselines in cross-shot consistency, visual quality, text alignment, and speech fidelity. Its world-model variant ranks first on WBench with an average score of 81.7 and achieves leading visual quality and long-horizon persistence on SANA-WM-Bench.