r-wom-world-model

name	r-wom-world-model
title	R-WoM: Retrieval-augmented World Model For Computer-use Agents
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2510.11892
keywords	["world-model","retrieval-augmentation","agent-planning","environment-simulation","computer-use"]
description	Ground LLM world models with retrieved current knowledge from tutorials and documentation. Reduce hallucination in environment prediction and improve long-horizon planning by 16-23% on web agent benchmarks.

R-WoM: Grounding Agent World Models with Retrieved Knowledge

LLMs used as world models for agent planning hallucinate and rely on stale training data, degrading performance on longer tasks. R-WoM grounds LLM simulations by retrieving current, factual knowledge from environment documentation, replacing reliance on memorized patterns with up-to-date information.

Core insight: world model accuracy depends on knowledge recency. By retrieving current documentation during simulation, agents make better predictions about environment dynamics, improving long-horizon planning accuracy where standard approaches compound errors.

Core Concept

Retrieved Context Grounding: Fetch relevant documentation/tutorials when simulating action outcomes, ensuring predictions reflect current environment state rather than LLM's training distribution.

Hybrid Simulation: Combine LLM reasoning (plan what to do) with retrieved facts (what will happen), separating reasoning from factual knowledge.

Architecture Overview

• Retriever: Searches documentation for relevant information
• World Model: LLM simulating environment given retrieved context
• Plan Executor: Uses world model predictions for decision-making
• Knowledge Base: Environment documentation/tutorials
• Long-Horizon Planner: Plans multi-step sequences

Implementation Steps

Stage 1: Set Up Knowledge Retrieval

Create retriever for environment documentation:

import torch
from transformers import AutoTokenizer, AutoModel
from sklearn.metrics.pairwise import cosine_similarity
import numpy as np

class EnvironmentRetriever:
    def __init__(self, documentation_path):
        """
        Initialize retriever for environment documentation.
        """

        self.tokenizer = AutoTokenizer.from_pretrained(
            'all-MiniLM-L6-v2'
        )

        self.embedding_model = AutoModel.from_pretrained(
            'all-MiniLM-L6-v2'
        )

        # Load and embed documentation
        self.documents = self.load_documentation(documentation_path)
        self.document_embeddings = self.embed_documents(
            self.documents
        )

    def load_documentation(self, doc_path):
        """
        Load environment documentation (tutorials, API docs, etc).
        """

        documents = []

        # Load from various sources
        with open(f"{doc_path}/api_reference.md") as f:
            api_docs = f.read().split('\n\n')
            documents.extend(api_docs)

        with open(f"{doc_path}/tutorials.md") as f:
            tutorials = f.read().split('\n\n')
            documents.extend(tutorials)

         documents

     ():
        

        embeddings = []

         doc  documents:
            tokens = .tokenizer(
                doc,
                truncation=,
                max_length=,
                return_tensors=
            )

             torch.no_grad():
                embedding = .embedding_model(**tokens)[].mean(dim=)

            embeddings.append(embedding.cpu().numpy())

         np.array(embeddings)

     ():
        

        
        query_tokens = .tokenizer(
            query,
            truncation=,
            max_length=,
            return_tensors=
        )

         torch.no_grad():
            query_embedding = .embedding_model(
                **query_tokens
            )[].mean(dim=).cpu().numpy()

        
        similarities = cosine_similarity(
            query_embedding,
            .document_embeddings
        )[]

        
        top_indices = np.argsort(similarities)[-k:][::-]
        top_docs = [.documents[i]  i  top_indices]
        top_scores = [similarities[i]  i  top_indices]

         top_docs, top_scores

Stage 2: Retrieval-Augmented World Model

Integrate retrieval into world model predictions:

class RetrievalAugmentedWorldModel:
    def __init__(self, retriever, world_model_name='llama-13b'):
        """
        Initialize world model with retrieval augmentation.
        """

        self.retriever = retriever

        self.world_model = AutoModelForCausalLM.from_pretrained(
            world_model_name
        )

        self.tokenizer = AutoTokenizer.from_pretrained(world_model_name)

    def predict_next_state(
        self,
        current_state,
        action,
        environment_name='web'
    ):
        """
        Predict next environment state given action.
        Retrieves documentation to ground prediction.
        """

        # Retrieve relevant documentation
        query = f"What happens when: {action}"

        retrieved_docs, scores = self.retriever.retrieve(
            query,
            k=3
        )

        # Construct prompt with retrieved context
        prompt = f"""
        Environment: {environment_name}

        Current state: {current_state}

        Relevant documentation:
        {self._format_documents(retrieved_docs)}

        Action: {action}

        Next state would be:
        """

        # Generate prediction
        input_ids = self.tokenizer.encode(
            prompt,
            return_tensors='pt'
        )

        with torch.no_grad():
            outputs = self.world_model.generate(
                input_ids,
                max_length=256,
                temperature=0.7
            )

        prediction = self.tokenizer.decode(outputs[])

         prediction

     ():
        

        formatted = 

         doc  documents:
            formatted += 

         formatted

     ():
        

        current_state = initial_state
        trajectory = [initial_state]
        predictions = []

         action  action_sequence:
            
            next_state = .predict_next_state(
                current_state,
                action,
                environment_name
            )

            trajectory.append(next_state)
            predictions.append(next_state)
            current_state = next_state

         trajectory, predictions

Stage 3: Long-Horizon Agent Planning

Use world model for multi-step planning:

class RetrievalAugmentedPlanner:
    def __init__(self, world_model):
        self.world_model = world_model

    def plan_trajectory(
        self,
        initial_state,
        goal,
        max_steps=10,
        environment_name='web'
    ):
        """
        Plan multi-step trajectory to reach goal.
        Uses world model predictions for planning.
        """

        plans = []
        plan_values = []

        # Generate multiple candidate plans
        for _ in range(3):
            plan = self.generate_candidate_plan(
                initial_state,
                goal,
                max_steps,
                environment_name
            )

            # Simulate plan to evaluate
            trajectory, predictions = (
                self.world_model.simulate_trajectory(
                    initial_state,
                    plan,
                    environment_name
                )
            )

            # Evaluate plan: does trajectory reach goal?
            plan_value = self.evaluate_plan(
                trajectory,
                goal
            )

            plans.append(plan)
            plan_values.append(plan_value)

        # Select best plan
        best_idx = np.argmax(plan_values)
        best_plan = plans[best_idx]

        return best_plan, plan_values[best_idx]

    def generate_candidate_plan(
        self,
        initial_state,
        goal,
        max_steps,
        environment_name
    ):
        """
        Generate action sequence plan.
        """

        prompt = f"""
        Environment: {environment_name}
        Initial state: {initial_state}
        Goal: 
        Max steps: 

        Generate a sequence of actions to reach the goal.
        Actions:
        """

        
        input_ids = .world_model.tokenizer.encode(
            prompt,
            return_tensors=
        )

         torch.no_grad():
            outputs = .world_model.world_model.generate(
                input_ids,
                max_length=
            )

        plan_text = .world_model.tokenizer.decode(outputs[])

        
        actions = .parse_actions(plan_text)

         actions

     ():
        

        final_state = trajectory[-]

        
        goal_achievement_score = 

         goal.lower()  final_state.lower():
            goal_achievement_score = 
        :
            
            goal_words = goal.lower().split()
            final_words = final_state.lower().split()

            matches = (
                  word  goal_words
                 word  final_words
            )

            goal_achievement_score = matches / (goal_words)

         goal_achievement_score

Practical Guidance

When to Use R-WoM:

• Agent tasks requiring knowledge of dynamic environments (web, APIs)
• Scenarios where environment documentation is available and up-to-date
• Long-horizon planning where hallucination compounds errors

When NOT to Use:

• Environments with no available documentation
• Tasks where domain knowledge is in training data (R-WoM won't help)
• Real-time planning where retrieval latency is prohibitive

Retrieval Configuration:

Aspect	Recommended	Rationale
Top-k documents	3-5	Balance context and noise
Embedding model	all-MiniLM	Fast, good quality
Refresh frequency	Per action	Keep state current

Typical Performance Improvements:

Benchmark	Baseline	R-WoM	Improvement
WebArena	20.3%	23.6%	+16.3%
OSWorld	28.1%	34.6%	+23.1%
Long-horizon (5+ steps)	15.2%	22.4%	+47%

Common Pitfalls:

• Documentation too generic (doesn't help planning)
• Retrieval ranking poor (irrelevant docs confuse model)
• Not updating documentation (stale knowledge)
• Too many retrieved documents (overwhelms context)

Reference

Based on the research at: https://arxiv.org/abs/2510.11892