DreamerV3-XP: Optimizing exploration through uncertainty estimation

arXiv 25.10 2025 52.9 method

TLDR

DreamerV3-XP enhances exploration via prioritized replay and ensemble disagreement intrinsic reward, tested on Atari and DMC tasks.

Reasoning

The paper presents a clear extension to DreamerV3 with two well-motivated exploration mechanisms and validates them on standard benchmarks, showing improvements in sparse-reward settings. However, the evaluation is limited to simulated environments and a subset of tasks, and the novelty is incremental rather than foundational.

Read-first score

Read-first score 52.9, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 39.

Recency 8%

86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark,result

Topical relevance 42%

55.7

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Field roles

Frontier

Rank sensitivity

Stability: volatile; rank range: 397.

Keyword Scores

world model

model-based reinforcement learning world model

world dynamics prediction

generative world model

world simulator

interactive world model

video world model

Deep Analysis

Innovations

Prioritized replay buffer scoring trajectories by return, reconstruction loss, and value error
Intrinsic reward based on disagreement over predicted environment rewards from an ensemble of world models

Methodology

DreamerV3-XP extends DreamerV3 by introducing a prioritized replay buffer that scores trajectories using return, reconstruction loss, and value error, and an intrinsic reward derived from the disagreement among an ensemble of world models predicting environment rewards. The method is evaluated on a subset of Atari100k and DeepMind Control Visual Benchmark tasks, comparing against the original DreamerV3.

Key Results

The extensions lead to faster learning and lower dynamics model loss, particularly in sparse-reward settings, while confirming the original DreamerV3 results.

Resources

Paper