EnvACE: Internalizing Environment Dynamics via World Rehearsal for Agentic Reinforcement Learning

arXiv 2026 46 method

TLDR

EnvACE replaces external environment interaction with world rehearsal, internalizing environment dynamics to train LLM agents, achieving strong performance across benchmarks.

Reasoning

The paper introduces a novel method for agentic RL that avoids costly environment interactions by having the policy simulate environment responses, which is a significant strength. The abstract provides evidence of effectiveness across multiple benchmarks and controlled studies, but does not discuss potential limitations or failure cases.

Read-first score

Read-first score 46, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 51.

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Topical relevance 29%

72.9

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,evaluation

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=code,github

Citation impact 18%

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. cited_by_count=0

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

Frontier

Rank sensitivity

Stability: volatile; rank range: 438.

Keyword Scores

world model

world dynamics prediction

model-based reinforcement learning world model

generative world model

interactive world model

world simulator

video world model

Deep Analysis

Innovations

World rehearsal: replacing external environment interaction with alternating acting and environment simulation, jointly optimized end-to-end
Internalized world model that supports private rehearsal at test time without additional external interaction

Methodology

EnvACE trains a policy that alternates between generating a tool call and then playing the environment role to produce the response, conditioning subsequent actions on the rehearsed response. Both roles are jointly optimized end-to-end using task-success rewards, internalizing action-response dynamics.

Key Results

EnvACE outperforms environment-scaling baselines on BFCL-v4, tau^2-Bench, VitaBench, and FinMCP-Bench, with consistent improvements across model scales. Test-time private rehearsal further boosts performance under a moderate budget.

Resources

Paper