Emergent Semantic Representations in World Models through Physical Interaction without Linguistic Supervision
TLDR
World models learn spatial semantic structure from physical exploration, organized by geometric principles, without language.
Reasoning
The paper presents a clear hypothesis and controlled experiments showing that VAE-based world models develop spatial semantic representations aligned with physical geometry, with strong quantitative evidence. However, it is limited to simulated environments and lacks real-world validation, and the scope is narrow.
Read-first score
Read-first score 61.2, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 39.
Field roles
Rank sensitivity
Stability: volatile; rank range: 404.
Keyword Scores
Deep Analysis
Innovations
- Demonstrates that world models develop spatial semantic representations from physical interaction without any linguistic supervision, organized by the geometric structure of the physical world.
- Identifies physical world geometry as the organizing principle of world model representations, supported by co-improvement of prediction and semantic alignment (Spearman r=-0.61, p=0.004).
- Provides causal evidence via a double knockout experiment: standard KL regularization (beta=0.1) collapses both prediction and semantic alignment, while reducing beta to 0.001 restores both, confirming the shared-driver account.
Methodology
A VAE-based world model is trained on random embodied exploration in a physical environment. The latent space is evaluated for spatial semantic structure using direction accuracy and position Representational Similarity Analysis (RSA) compared to randomly initialized encoders. A double knockout experiment manipulates the KL regularization coefficient (beta) to test the shared-driver account between prediction performance and semantic alignment.
Key Results
The trained world model achieves direction accuracy 0.677±0.029 vs. 0.547 for random encoders, and position RSA 0.192±0.047 vs. 0.029 (6.6x improvement). Prediction performance and semantic alignment co-improve over training (Spearman r=-0.61, p=0.004), and both collapse simultaneously under standard KL regularization (beta=0.1) by step 50,000, while reducing beta to 0.001 restores both.
Limitations
- The study focuses on spatial semantic structure (direction and position) and does not address other types of semantic concepts (e.g., object categories, affordances).
- Training is based on random embodied exploration, which may not capture the structured exploration strategies typical of real-world agents.
- The VAE-based architecture and specific hyperparameter choices (e.g., beta values) may limit generalizability to other world model designs or environments.
- The experiments are likely conducted in simulation; transfer to real-world physical interaction is not demonstrated.