⌁ Awesome World Model Hub Papers · Datasets · Projects
中文 GitHub
← Back to papers

Dyna-Think: Synergizing Reasoning, Acting, and World Model Simulation in AI Agents

arXiv 25.5 2025 55.9 method

TLDR

Dyna-Think integrates planning with an internal world model, reasoning, and acting to improve AI agent performance via imitation and dyna training.

Reasoning

The paper proposes a novel framework combining world model simulation with reasoning and acting, validated on real-world benchmarks with improved token efficiency. However, the abstract lacks details on world model architecture and generalizability beyond the tested environments.

Read-first score

Read-first score 55.9, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 44.

Recency 8%
86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Topical relevance 42%
62.9

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Methodology quality 25%
60

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=evaluation,result

Reproducibility 25%
30

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Field roles

Frontier

Rank sensitivity

Stability: volatile; rank range: 359.

Keyword Scores

world model
10
model-based reinforcement learning world model
9
world dynamics prediction
8
world simulator
7
interactive world model
6
generative world model
4
video world model
0

Deep Analysis

Innovations

  • Proposes Dyna-Think framework that integrates planning with an internal world model, reasoning, and acting for AI agents.
  • Introduces Dyna-Think Imitation Learning (DIT) to initialize policy by reconstructing R1's thinking process to focus on world model simulation relevant to actions.
  • Introduces Dyna-Think Dyna Training (DDT) with two-stage training: first improve world modeling via state prediction or critique generation, then improve action via policy training.
  • Demonstrates that using critique generation for world model training is effective for improving policy performance.
  • Shows correlation between better world modeling abilities and better agent performance.

Methodology

The paper proposes Dyna-Think, a thinking framework that integrates planning with an internal world model, reasoning, and acting. To enable it, they propose Dyna-Think Imitation Learning (DIT) which reconstructs the thinking process of R1 to focus on world model simulation relevant to actions, and Dyna-Think Dyna Training (DDT) which uses a two-stage process: first improving world modeling via state prediction or critique generation, then improving action via policy training. Evaluation is conducted on OSWorld and WindowsAgentArena, comparing best-of-n performance and token efficiency against R1.

Key Results

Dyna-Think improves in-domain and out-of-domain performance, achieving similar best-of-n performance compared to R1 while generating 2x less tokens on average. Additionally, critique generation for world model training is effective, and better world modeling correlates with better agent performance.

Resources

Paper

Tags