WorldSonus: Bringing Sound to Worlds

arXiv 2026 32.3 method

TLDR

WorldSonus generates real-time, interactive, spatially aligned audio for visual world models using streaming causal autoregressive diffusion, matching open-domain benchmarks.

Reasoning

The paper addresses a novel and underexplored problem of adding sound to world models, with strong emphasis on real-time efficiency, interactive control, and spatial alignment. Its strengths are clear problem framing and demonstrated generalization, though the abstract lacks quantitative experimental details and the connection to core world-model prediction is mainly as an application context.

Read-first score

Read-first score 32.3, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 33.

Recency 5%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Methodology quality 15%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark,experiment

Topical relevance 25%

47.1

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 15%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=github

Graph prestige 15%

5.7

PageRank-style prestige over the local citation graph built from available OpenAlex references. local_pagerank=0.0016

Citation impact 15%

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. cited_by_count=0

Citation velocity 10%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

Frontier

Rank sensitivity

Stability: volatile; rank range: 227.

Keyword Scores

world model

interactive world model

video world model

generative world model

world simulator

world dynamics prediction

model-based reinforcement learning world model

Deep Analysis

Innovations

Streaming causal autoregressive diffusion architecture that synthesizes audio chunks in real time with a low real-time factor of 0.41
Audio-centric captioning pipeline with chunk-indexed prompt scheduling for interactive mid-stream control of sound events
High-quality stereo supervision curated from diverse stereo and ambisonic data for spatially aligned stereo synthesis

Methodology

WorldSonus is an interactive video-to-audio framework that uses a streaming causal autoregressive diffusion model to generate audio chunks in real time. It incorporates an audio-centric captioning pipeline with chunk-indexed prompt scheduling for dynamic, mid-stream sound control and leverages stereo supervision from diverse stereo and ambisonic data for spatial alignment. The approach is evaluated on open-domain video-to-audio benchmarks against state-of-the-art bidirectional models.

Key Results

WorldSonus achieves a real-time factor of 0.41 for audio generation and matches or outperforms state-of-the-art bidirectional models on open-domain video-to-audio benchmarks in both acoustic quality and spatial alignment.

Resources

Paper