⌁ Awesome World Model Hub Papers · Datasets · Projects
中文 GitHub
← Back to papers

Task Aware Dreamer for Task Generalization in Reinforcement Learning

arXiv 23.3 2023 56.1 method

TLDR

Introduces Task Aware Dreamer (TAD) that integrates reward-informed features into world models for task generalization in reinforcement learning.

Reasoning

The paper presents a novel method (TAD) that extends world models to handle task generalization by incorporating reward information, supported by a new metric (TDR) and theoretical justification. However, the experiments are limited to simulated environments, and the approach assumes similar dynamics across tasks, which may restrict real-world applicability.

Read-first score

Read-first score 56.1, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 39.

Methodology quality 25%
80

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,experiment,metric

Recency 8%
65.1

Uses a gentle age decay so recent papers surface without erasing older foundations. 2023

Topical relevance 42%
55.7

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 25%
30

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Field roles

Methodology anchor

Rank sensitivity

Stability: volatile; rank range: 132.

Keyword Scores

world model
10
model-based reinforcement learning world model
8
generative world model
7
world dynamics prediction
6
world simulator
5
interactive world model
2
video world model
1

Deep Analysis

Innovations

  • Demonstrates that training a general world model can leverage similar structures across tasks to improve generalization.
  • Introduces Task Aware Dreamer (TAD), which integrates reward-informed features to identify consistent latent characteristics across tasks.
  • Computes a variational lower bound with a new term that differentiates tasks using their states as the optimization objective.
  • Proposes Task Distribution Relevance (TDR) metric to quantitatively measure the relevance of different tasks.
  • Shows that Markovian policies struggle to distinguish tasks with high TDR, motivating the need for reward-informed policies in TAD.

Methodology

TAD extends world models to task generalization by incorporating reward-informed features into the latent representation. It optimizes a variational lower bound on the log-likelihood of sample data, adding a term that differentiates tasks based on states. The method is evaluated on both image-based and state-based tasks, comparing against baselines (implied) and using the TDR metric to analyze task relevance.

Key Results

Extensive experiments show that TAD significantly improves performance when handling different tasks simultaneously, especially those with high TDR, and demonstrates strong generalization ability to unseen tasks.

Limitations

  • The method requires access to reward functions during training to incorporate reward-informed features, which may not be available in all settings.

Resources

Paper

Tags