web-coach-self-evolving-agents

name	web-coach-self-evolving-agents
title	WebCoach: Self-Evolving Web Agents with Cross-Session Memory
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2511.12997
keywords	["Web Agents","Episodic Memory","Self-Evolution","Cross-Session Learning","Runtime Guidance"]
description	Enable web agents to improve over time by curating episodic memory from navigation trajectories and dynamically injecting task-specific guidance at runtime—no retraining required, persistent improvement across sessions.

Enable Web Agents to Self-Improve via Cross-Session Episodic Memory

Web agents (browser automation, form filling, multi-step navigation) often fail on edge cases or novel sites. Retraining on every failure is expensive. WebCoach enables self-evolution without retraining: the agent maintains an External Memory Store (EMS) of past navigation episodes, continuously refining its decision-making by retrieving and injecting relevant past experiences at runtime.

A Coach component evaluates the agent's current trajectory and injects guidance when it detects failure risk or identifies superior strategies in memory. This achieves persistent agent improvement across sessions without model updates—learning happens through retrieval and composition, not gradient descent.

Core Concept

Web agents typically operate in isolation: each session is independent, successful and failed trajectories are discarded. WebCoach breaks this pattern by maintaining a shared episodic memory across sessions. When an agent encounters a new task, the Coach retrieves similar past episodes and injects relevant guidance into the agent's context.

The system comprises three components working in tandem:

1. WebCondenser: Summarizes raw navigation logs into structured episodes with embeddings, success labels, and error patterns
2. External Memory Store (EMS): Persists completed episodes with semantic search capability
3. Coach: LLM-based runtime decision engine that retrieves relevant episodes and injects guidance when needed

Together, they enable agents to learn from cross-session experience without modifying the underlying policy.

Architecture Overview

• Navigation Trajectory Logging: Capture observation-action-reward sequences; store only completed episodes (success or failure endpoints)
• WebCondenser: Convert raw logs to structured episodes with (summary, embedding, success_label, error_patterns)
• External Memory Store (EMS): Semantic index enabling fast retrieval of similar past episodes; stores up to millions of episodes
• Coach Decision Engine: Evaluate current trajectory; retrieve top-k similar episodes; decide if intervention needed; inject guidance as system message
• No-Retraining Integration: Coach advice appends to message history; agent processes without policy modification

Implementation Steps

Step 1: WebCondenser—Normalize Navigation Logs. Convert raw traces to structured episodes.

class WebCondenser:
     ():
        .llm = load_small_llm(llm_model)  
        .embedding_model = load_embedding_model()
        .embedding_dim = embedding_dim

     ():
        
        
        narrative = ._format_trajectory_text(trajectory)

        
        summary = .llm.summarize(
            narrative,
            max_tokens=,
            system_prompt=
        )

        
        final_reward = trajectory[-][]
        success = final_reward > 

        
        error_patterns = ._extract_error_patterns(trajectory)

        
        embedding = .embedding_model.encode(summary)  

         {
            : summary,
            : embedding,
            : success,
            : error_patterns,
            : ((trajectory))  
        }

     ():
        
        lines = []
         obs, action, reward  trajectory:
            lines.append()
            lines.append()
            lines.append()
         .join(lines)

     ():
        
        patterns = []

         obs, action, reward  trajectory:
             reward < :  
                
                action_type = action.split()[]  action  
                patterns.append({
                    : action_type,
                    : obs,
                    : obs
                })

        
         patterns[:]