Train autoregressive world models on 1M+ real web interactions for accurate browser state prediction. Enables agent training with 100× more data than prior approaches, achieving GPT-4o comparable performance with format flexibility and cross-domain generalization.
WebWorld: A Large-Scale World Model for Web Agent Training
version
0.0.2
engine
skillxiv-v0.0.2-claude-opus-4.6
license
MIT
url
https://arxiv.org/abs/2602.14721
keywords
["World Models","Web Agents","Simulation","Large-Scale Training Data","Generalization"]
description
Train autoregressive world models on 1M+ real web interactions for accurate browser state prediction. Enables agent training with 100× more data than prior approaches, achieving GPT-4o comparable performance with format flexibility and cross-domain generalization.
WebWorld: Large-Scale Web Agent Training Environment
Problem Context
Web agent training requires massive interaction data that's expensive or impossible to collect at scale. Existing web world models use sandbox or closed environments with limited realism. Data scarcity forces agents to overfit to specific websites. Prior approaches generated only 10K-100K trajectories. Standard models trained on such small datasets don't capture real web complexity, limiting agent performance and generalization.
Core Concept
WebWorld is a large-scale autoregressive world model trained on over 1 million real-world web interaction trajectories. Rather than generating synthetic data or using sandboxes, it learns from authentic web behavior patterns. The key innovation: collecting data through three complementary strategies (randomized crawling, autonomous exploration, task-oriented execution) creates diverse, realistic training distributions.
The model predicts next browser state given instruction and interaction history, enabling agents to train entirely in simulation. Multiple format support (A11y Trees, HTML, XML, Markdown) and explicit reasoning injection through chain-of-thought enable strong generalization to code, GUI, and game environments.
Architecture Overview
Autoregressive State Predictor: Predicts next DOM/browser state given action and history
classAutoregressiveWorldModel(nn.Module):
"""
Predict next browser state given current state and action.
Trained on 1M+ real web interaction trajectories.
"""def__init__(self, vocab_size, hidden_dim=2048, num_layers=24,
model_size='14B'):
super().__init__()
self.hidden_dim = hidden_dim
self.model_size = model_size
# Transformer backbone for state predictionself.backbone = TransformerLM(
vocab_size=vocab_size,
hidden_dim=hidden_dim,
num_layers=num_layers,
num_heads=32
)
# Action encoderself.action_encoder = nn.Embedding(num_actions := 1000,
hidden_dim)
defpredict_next_state(self, current_state, action,
interaction_history, format='html'):
"""
Predict next browser state in specified format.
Supports A11y Trees, HTML, XML, Markdown.
"""# Tokenize current state
state_tokens = tokenize_state(current_state, format)
# Encode action
action_embedding = self.action_encoder(action)
# Append action to state sequence
full_sequence = torch.cat([
state_tokens,
action_embedding.unsqueeze(0)
], dim=0)
# Generate next state tokens autoregressivelywith torch.no_grad():
logits = self.backbone(full_sequence)
# Sample next state tokens
next_state_tokens = sample_next_tokens(
logits, temperature=0.7, max_tokens=512)
# Decode tokens to state
next_state = decode_state(next_state_tokens, format)
return next_state
defgenerate_trajectory(self, initial_state, instruction,
max_steps=30, format='html'):
"""
Generate complete trajectory given initial state and instruction.
"""
trajectory = {
'initial_state': initial_state,
'instruction': instruction,
'interactions': [],
'states': [initial_state]
}
current_state = initial_state
for step inrange(max_steps):
# Decide action based on instruction and current state
action = decide_action(current_state, instruction)
# Predict next state
next_state = self.predict_next_state(
current_state, action, trajectory['interactions'],
format=format)
trajectory['interactions'].append({
'action': action,
'from_state': current_state,
'to_state': next_state
})
trajectory['states'].append(next_state)
current_state = next_state
return trajectory
Chain-of-thought synthesis for reasoning:
classReasoningInjection:
"""
Inject explicit reasoning through chain-of-thought synthesis.
Improves reasoning capability without requiring massive annotation.
"""def__init__(self, cot_generator, sample_size=1000):
self.cot_gen = cot_generator
self.sample_size = sample_size
definject_reasoning_into_dataset(self, trajectories):
"""
Add CoT reasoning to subset of trajectories.
Improves reasoning without expensive full annotation.
"""# Sample subset for CoT synthesis
sample = random.sample(trajectories, self.sample_size)
annotated_trajectories = []
for traj in sample:
# Generate reasoning for trajectory
state = traj['interactions'][0]['before_state']
action = traj['interactions'][0]['action']
# Generate CoT explanation
reasoning = self.cot_gen.explain_action(
state, action, traj.get('instruction', ''))
# Add reasoning to trajectory
annotated_traj = copy.deepcopy(traj)
annotated_traj['reasoning'] = reasoning
annotated_trajectories.append(annotated_traj)
return annotated_trajectories
Multi-format state representation:
classMultiFormatStateRepresentation:
"""
Support multiple state formats for flexibility.
Enables transfer to code, GUI, game environments.
""" @staticmethoddefstate_to_a11y_tree(dom_tree):
"""Convert DOM to accessibility tree (high-level structure)."""
a11y = {
'type': 'root',
'children': [],
'text': ''
}
# Traverse DOM and build accessibility structurereturn a11y
@staticmethoddefstate_to_html(dom_tree):
"""Convert to HTML representation."""return dom_to_html_string(dom_tree)
@staticmethoddefstate_to_markdown(dom_tree):
"""Convert to readable Markdown."""
markdown = []
for element in traverse_dom(dom_tree):
if element.tag == 'h1':
markdown.append(f"# {element.text}")
elif element.tag == 'button':
markdown.append(f"[Button: {element.text}]")
return'\n'.join(markdown)
@staticmethoddefstate_to_xml(dom_tree):
"""Convert to XML representation."""return dom_to_xml_string(dom_tree)
Practical Guidance
When to use:
Training web agents at scale
Need diverse, realistic interaction data
Want agents that generalize across websites
Have limited real API/browser access for training
Data collection strategy:
Level 1 (Randomized): 60-70% of total
Fast to collect
Natural interaction patterns
Budget: minimal (automated)
Level 2 (Autonomous): 10-15% of total
Agent-driven objectives
Diverse exploration strategies
Budget: moderate (agent overhead)
Level 3 (Task-Oriented): 20-30% of total
Structured objectives
Higher quality signal
Budget: higher (task synthesis/validation)
Model sizing recommendations:
8B: Fast training/inference, good for simple tasks
14B: Balanced (recommended for most applications)
32B: Highest quality, slower inference
Format selection:
A11y Trees: Fast processing, good structure extraction
HTML: Detailed, but verbose
Markdown: Concise, human-readable
XML: Structured, good for element relationships
Use multiple formats during training for robustness
Reasoning injection:
Annotate 5-10% of training data with CoT reasoning
Focus on complex state transitions
Use GPT-4 or similar for high-quality CoT
Include in trajectory dataset for training
Expected performance:
WebArena benchmark: +9.2% over baselines
GPT-4o comparable performance on many tasks
Strong generalization to code, GUI, games
30+ step trajectories maintain coherence
1M trajectories approach 50 billion tokens
Training considerations:
Batch size: 256-512 (depending on hardware)
Learning rate: 1e-4 for fine-tuning, 1e-5 for continued pre-training
Warmup steps: 5,000-10,000
Eval frequency: every 5,000 steps
Total training: 100-200 hours on 8xH100
Agent fine-tuning on WebWorld:
Initialize agent with language model weights
Fine-tune on WebWorld-generated trajectories
Use behavior cloning for 1,000-5,000 steps
Then fine-tune with RL on real tasks
Achieves competitive performance with 10× less real data
Reference
Large-scale autoregressive world models trained on authentic web interactions enable efficient agent training in simulation. By combining randomized, autonomous, and task-oriented data collection with multi-format representation and reasoning injection, WebWorld creates a generalizable training environment that captures real web complexity at scale.
