R-AIF: Solving Sparse-Reward Robotic Tasks from Pixels with Active Inference and World Models

arXiv 24.9 2024 44.5 method, application

TLDR

R-AIF uses active inference and world models to solve sparse-reward robotic tasks from pixels, outperforming SOTA in POMDPs.

Reasoning

The paper introduces novel prior preference learning and self-revision schedules for active inference in continuous-action POMDPs with sparse rewards, showing empirical improvements. However, the abstract does not specify real-world robotic experiments, and the reliance on simulated environments may limit generalizability.

Read-first score

Read-first score 44.5, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 24.

Recency 8%

75.1

Uses a gentle age decay so recent papers surface without erasing older foundations. 2024

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=result

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=code,github

Topical relevance 42%

34.3

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Field roles

Candidate

Rank sensitivity

Stability: volatile; rank range: 206.

Keyword Scores

world model

model-based reinforcement learning world model

world dynamics prediction

generative world model

world simulator

interactive world model

video world model

Deep Analysis

Innovations

Novel prior preference learning techniques
Self-revision schedules for active inference in POMDPs

Methodology

The paper proposes R-AIF, an active inference agent that uses world models to handle partially observable Markov decision processes (POMDPs) with pixel observations. It introduces novel prior preference learning techniques and self-revision schedules to enable the agent to excel in sparse-reward, continuous action, goal-based robotic control environments.

Key Results

Empirically, the agents demonstrate improved performance over state-of-the-art models in terms of cumulative rewards, relative stability, and success rate.

Resources

Paper