COMAP: Co-Evolving World Models and Agent Policies for LLM Agents

arXiv 2026 68.1 method

TLDR

COMAP co-evolves textual world models and agent policies via closed-loop interaction, improving decision-making across embodied, web, and tool-use tasks.

Reasoning

The paper presents a novel framework that dynamically updates world models based on agent trajectories, addressing limitations of fixed models and external rewards. Strengths include clear methodology and strong empirical results across multiple benchmarks; weaknesses are not evident from abstract alone but the focus on textual models may limit applicability to visual domains.

Read-first score

Read-first score 68.1, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 46.

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Citation impact 18%

91.8

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. citation_normalized_percentile=0.91829655

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=True; dataset=False; markers=code,github

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,benchmark,result

Topical relevance 29%

65.7

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FoundationFrontierBridgeMethodology anchorReproducibility anchor

Rank sensitivity

Stability: volatile; rank range: 424.

Keyword Scores

world model

world dynamics prediction

interactive world model

model-based reinforcement learning world model

world simulator

generative world model

video world model

Deep Analysis

Innovations

Co-evolution of textual world models and agent policies through closed-loop interaction, enabling adaptation to on-policy state-action distributions.
Future-aware reflection mechanism where the agent estimates reliability of world model predictions and refines actions accordingly.
Self-distillation update of the world model using on-policy trajectories to match the agent's evolving interaction distribution.

Methodology

COMAP proposes a framework where at each decision step, a textual world model predicts future state feedback for candidate actions, and the agent performs future-aware reflection by estimating the reliability of this feedback and refining its action. The resulting on-policy trajectories are used to update the world model via self-distillation, creating a closed-loop co-evolutionary process. The framework is evaluated across embodied task planning, Web navigation, and tool-use benchmarks against competitive baselines.

Key Results

COMAP consistently outperforms competitive baselines, achieving a +16.75% relative improvement with Qwen3-4B. Further analyses show that the co-evolutionary loop improves the world model's prediction accuracy over time and leads to more effective long-horizon decision-making.

Resources

Paper Code