Dreaming Of Others: Latent Teammate Modeling In World Models For Multi-Agent Reinforcement Learning
TLDR
Proposes factoring teammates into world models for multi-agent RL using Theory-of-Mind to enable zero-shot coordination.
Reasoning
The paper introduces a novel architecture that integrates teammate modeling into Dreamer-style world models, addressing a key limitation in MARL. However, it is a proposal without empirical results, relying on outlined benchmarks rather than demonstrated performance.
Read-first score
Read-first score 63.5, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 43.
Field roles
Rank sensitivity
Stability: volatile; rank range: 396.
Keyword Scores
Deep Analysis
Innovations
- Treating teammates as structured, learnable components within the agent's world model, rather than as part of the environment dynamics.
- Factorization of the Dreamer-style recurrent state-space model (RSSM) latent state into separate environment and teammate components.
- Introduction of an auxiliary Theory-of-Mind (ToM) head to infer latent embeddings of partner behavior (character, intent, predicted actions) from partial trajectories.
- Conditioning the actor and critic on inferred teammate latents to enable zero-shot and few-shot coordination in partially observable settings.
Methodology
The paper proposes an architecture that extends Dreamer-style world models to multi-agent settings by factorizing the latent state of an RSSM into environment and teammate components. An auxiliary Theory-of-Mind head is learned to infer latent embeddings of partner behavior from partial trajectories, and these teammate latents condition both the actor and critic networks, allowing the agent to imagine and adapt to diverse collaborators. The approach is evaluated through proposed benchmarks and evaluation protocols for zero-shot and few-shot coordination.
Key Results
The paper outlines how the proposed approach can support zero-shot and few-shot coordination in partially observable settings, but no empirical results are presented; the work is a conceptual proposal with a set of benchmarks and evaluation protocols suggested for future assessment.
Limitations
- The paper is a conceptual proposal without empirical validation or experimental results.
- The approach assumes that teammate latents can be reliably inferred from partial trajectories, which may be challenging in highly stochastic or adversarial settings.
- The factorization and ToM head may introduce additional computational complexity and training difficulty.
- The method is described only for cooperative multi-agent settings; applicability to competitive or mixed-motive scenarios is not addressed.
- Scalability to large numbers of agents or complex teammate behaviors is not discussed.