Masked Diffusion Language Models are Strong and Steerable Text-Based World Models for Agentic RL

arXiv 2026 52.9 method

TLDR

Masked diffusion language models outperform larger autoregressive models as steerable text-based world models for agentic RL, with better coherence and diversity.

Reasoning

The paper introduces a novel steerable text-based world model using masked diffusion language models, with strong empirical evidence across multiple environments and agent backbones. However, its focus on text-based domains limits generalizability to other modalities like video, and the reliance on curated trajectories may not capture all real-world dynamics.

Read-first score

Read-first score 52.9, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 54.

Methodology quality 18%

100

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=ablation,analysis,baseline,dataset,evaluation

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Topical relevance 29%

77.1

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=dataset

Citation impact 18%

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. cited_by_count=0

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 753.

Keyword Scores

world model

world dynamics prediction

world simulator

generative world model

model-based reinforcement learning world model

interactive world model

video world model

Deep Analysis

Innovations

Formalizing text-based world modeling as a steerable transition-dynamics problem decomposed into initial state, task context, tool schemas, domain rules, and steering directives.
Proposing masked diffusion language models (MDLMs) as bidirectional, anchor-aware world models to overcome the left-to-right bias of autoregressive models.
Introducing a plug-and-play GRPO training framework with deterministic state checks.
Curating a dataset of 239,403 grounded state-action trajectories from nine open-source environments and twelve frontier model families.
Demonstrating that MDLMs achieve better coherence, groundedness, and rollout diversity than LLMs over 4x their size, and up to 47% absolute gain in zero-shot OOD transfer.

Methodology

The authors formalize text-based world modeling as a steerable transition-dynamics problem, then curate a large dataset of grounded trajectories. They compare autoregressive language models and masked diffusion language models (MDLMs) as world models, train with a GRPO framework incorporating deterministic state checks, and evaluate zero-shot transfer on three OOD environments (ScienceWorld, ALFWorld, AppWorld) using three agent backbones (LFM2.5, Qwen3, Mistral).

Key Results

MDLMs outperform LLMs over 4x their parameter size in coherence, groundedness, and rollout diversity with comparable inference latency, and achieve up to 47% absolute gains over baselines in zero-shot transfer to OOD environments without environment-specific fine-tuning.

Limitations

Behavioral analysis reveals failure modes under adversarial scenarios.

Resources

Paper