Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation

arXiv 24.10 2024 52.5 method, application

TLDR

Proposes a world-model-augmented web agent that simulates action outcomes to improve decision-making, validated on WebArena and Mind2Web.

Reasoning

The paper clearly identifies a gap (LLMs lack world models) and introduces a novel method with transition-focused observation abstraction. Strengths include empirical validation on real benchmarks and cost-efficiency. Weaknesses are the narrow focus on web navigation and potential scalability issues.

Read-first score

Read-first score 52.5, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 40.

Recency 8%

75.1

Uses a gentle age decay so recent papers surface without erasing older foundations. 2024

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark,experiment

Topical relevance 42%

57.1

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Field roles

Candidate

Rank sensitivity

Stability: volatile; rank range: 354.

Keyword Scores

world model

world dynamics prediction

interactive world model

world simulator

generative world model

model-based reinforcement learning world model

video world model

Deep Analysis

Innovations

Identifying the absence of world models in current LLMs (e.g., GPT-4o, Claude-3.5-Sonnet) through preliminary analyses
Proposing a World-model-augmented (WMA) web agent that simulates action outcomes to improve decision-making
Introducing a transition-focused observation abstraction that uses free-form natural language descriptions of key state differences to train LLMs as world models for predicting next observations

Methodology

The study first conducts preliminary analyses to confirm the lack of world models in current LLMs. It then proposes a World-model-augmented (WMA) web agent that simulates the outcomes of its actions. To overcome challenges in training LLMs as world models for predicting next observations (e.g., repeated elements, long HTML), it introduces a transition-focused observation abstraction that outputs free-form natural language descriptions highlighting important state differences between time steps. The agent is evaluated on WebArena and Mind2Web benchmarks.

Key Results

The world models improve agents' policy selection without additional training, and the WMA agent demonstrates cost- and time-efficiency compared to recent tree-search-based agents.

Resources

Paper