AcceRL: A Distributed Asynchronous Reinforcement Learning and World Model Framework for Vision-Language-Action Models

arXiv 26.3 2026 35.5 system, application

TLDR

AcceRL is a distributed asynchronous RL framework for VLA models that improves throughput and sample efficiency by isolating rollouts, inference, and updates, and integrating world models.

Reasoning

The paper presents a practical solution to synchronization bottlenecks in RL for VLA models, achieving notable throughput and sample efficiency gains. However, the world model integration is not a novel contribution, and experiments are limited to simulation benchmarks without real-world validation.

Read-first score

Read-first score 35.5, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 11.

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,experiment

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=code,github

Citation impact 18%

35.5

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. citation_normalized_percentile=0.35463074

Topical relevance 29%

15.7

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

Frontier

Rank sensitivity

Stability: volatile; rank range: 93.

Keyword Scores

world model

model-based reinforcement learning world model

world simulator

generative world model

interactive world model

video world model

world dynamics prediction

Deep Analysis

Innovations

Distributed asynchronous RL framework that physically isolates environment rollouts, model inference, and gradient updates to eliminate cascading long-tail idle bubbles
Modular design supporting plug-and-play integration of diverse world models into the distributed pipeline
Achieves up to 200x improvement in online sample efficiency by leveraging a world model pre-trained on 1,000 offline trajectories

Methodology

AcceRL is a distributed asynchronous reinforcement learning framework for Vision-Language-Action (VLA) models. It physically separates environment rollouts, model inference, and gradient updates across different workers to maximize hardware utilization and avoid synchronization barriers. The framework also features a modular design that allows seamless integration of pre-trained world models into the distributed pipeline.

Key Results

The asynchronous architecture delivers a 2.4x throughput speedup over leading synchronous baselines. By leveraging a world model pre-trained on 1,000 offline trajectories, AcceRL achieves up to a 200x improvement in online sample efficiency on LIBERO-Spatial, while maintaining competitive performance across all four LIBERO task suites.

Resources

Paper