SafeDreamer: Safe Reinforcement Learning with World Models

ICLR 24 2024 44.6 method

TLDR

SafeDreamer integrates Lagrangian methods into Dreamer world models for safe RL, achieving near-zero cost on Safety-Gymnasium benchmarks.

Reasoning

The paper's strength lies in combining world models with safety constraints to improve sample efficiency and safety in RL, demonstrated on standard benchmarks. Weaknesses include lack of real-world validation and limited discussion of failure modes or scalability.

Read-first score

Read-first score 44.6, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 39.

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=True; dataset=False; markers=code,github,repository

Recency 6%

75.1

Uses a gentle age decay so recent papers surface without erasing older foundations. 2024

Topical relevance 29%

55.7

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark

Citation impact 18%

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. cited_by_count=0

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

Reproducibility anchor

Rank sensitivity

Stability: volatile; rank range: 483.

Keyword Scores

world model

model-based reinforcement learning world model

world dynamics prediction

generative world model

world simulator

interactive world model

video world model

Deep Analysis

Innovations

Incorporating Lagrangian-based methods into world model planning processes within the Dreamer framework
Achieving nearly zero-cost performance on vision-only safety tasks in the Safety-Gymnasium benchmark

Methodology

SafeDreamer integrates Lagrangian-based methods into the world model planning processes of the Dreamer framework. It is evaluated on the Safety-Gymnasium benchmark across low-dimensional and vision-only input tasks, aiming to balance performance and safety.

Key Results

SafeDreamer achieves nearly zero-cost performance on various tasks in the Safety-Gymnasium benchmark, including vision-only tasks, demonstrating efficacy in balancing performance and safety.

Resources

Paper Code