Grounded Answers for Multi-agent Decision-making Problem through Generative World Model

NeurIPS 24 2024 68.3 method

TLDR

Integrates a language-guided generative world model into multi-agent RL to produce grounded answers for decision-making problems.

Reasoning

The paper presents a novel framework combining a language-guided simulator with separate dynamics and reward models, showing strong empirical results on the StarCraft Multi-Agent Challenge. However, it is evaluated on only one benchmark, limiting generalizability claims.

Read-first score

Read-first score 68.3, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 58.

Topical relevance 42%

82.9

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark,evaluation,result

Recency 8%

75.1

Uses a gentle age decay so recent papers surface without erasing older foundations. 2024

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Field roles

Methodology anchor

Rank sensitivity

Stability: volatile; rank range: 191.

Keyword Scores

world model

generative world model

world simulator

model-based reinforcement learning world model

world dynamics prediction

interactive world model

video world model

Deep Analysis

Innovations

Integration of a language-guided simulator (world model) into multi-agent reinforcement learning pipeline to generate grounded answers
Separate learning of dynamics and reward models, with dynamics using an image tokenizer and causal transformer for autoregressive interaction transitions
Reward model as a bidirectional transformer trained by maximizing likelihood of expert demonstration trajectories under language guidance
Generation of image sequences as answers by running the converged policy on the dynamics model

Methodology

The proposed framework uses a world model that separately learns dynamics (via an image tokenizer and causal transformer) and reward (via a bidirectional transformer trained with language-guided likelihood maximization on expert demonstrations). Given a current state image and task description, the world model is used to train a joint policy, and the converged policy is run on the dynamics model to produce an image sequence as the answer.

Key Results

The framework achieves superior performance on both training and unseen tasks of the StarCraft Multi-Agent Challenge benchmark, generating consistent interaction sequences and explainable reward functions at interaction states.

Limitations

Evaluation is limited to the StarCraft Multi-Agent Challenge benchmark; generalization to other multi-agent decision-making domains is not demonstrated
The method requires language guidance for the reward model, which may not be available or easily defined in all environments
The separate training of dynamics and reward models may introduce additional computational complexity and training overhead

Resources

Paper