memory-indexed-experience-scaling

name	memory-indexed-experience-scaling
title	Memex(RL): Scaling Long-Horizon LLM Agents via Indexed Experience Memory
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2603.04257
keywords	["Long-Horizon Reasoning","Memory Management","Context Compression","Agent Scaling","Reinforcement Learning"]
description	Enable long-horizon agents to manage finite context by separating working memory from persistent storage. Use indexed summaries with pointers to archived evidence, treating memory operations as first-class agent actions learned via RL.

Memex(RL): Indexed Experience Memory for Scaling Long-Horizon Agents

Long-horizon multi-step reasoning tasks overwhelm finite context windows: agents either repeat full conversation history (inefficient) or use lossy summaries (information loss). Memex introduces a memory architecture where agents explicitly manage both working context and long-term storage through learned operations. The system maintains compact in-context indexed summaries with pointers to externally archived artifacts, enabling full-fidelity evidence retrieval on demand.

The core innovation treats memory as a first-class agent capability: read operations dereference indices to retrieve exact past information; write operations compress state into indexed summaries. This mirrors how humans manage complex projects with external notes and references while keeping working memory focused.

Core Concept

Memex decomposes memory into two complementary systems:

1. Working Context: Compact in-context summary (~1-2KB) containing indexed references to past interactions
2. Experience Store: External archive of full-fidelity artifacts (full trajectories, tool outputs, decision trees) under stable indices

Agents learn to:

• Write: Compress key information into indexed summary for future reference
• Read: Dereference indices to retrieve exact past information when needed
• Manage: Decide what to compress, when to retrieve, based on task demands

This separation enables agents to maintain reasoning coherence over thousands of tokens while staying within practical context limits.

Architecture Overview

• Input: Long-horizon task with multi-step subtasks
• Working Context Module: Compressed summary with embedded indices
• Experience Store: Persistent key-value archive indexed by discrete IDs
• Read/Write Operators: Agent-learned memory operations
• Action Space: Extended with memory operations (read_idx, write_summary)
• Output: Decision with access to complete historical context

Implementation Steps

Step 1: Design experience store and indexing scheme

Create a persistent storage system for archived experiences with stable access keys.

class ExperienceStore:
    """External memory store for agent artifacts."""
     ():
        .store = {}
        .index_counter = 
        .max_size = max_size

     ():
        
         (.store) >= .max_size:
            
            oldest_idx = (.store.keys(),
                            key= k: .store[k][])
             .store[oldest_idx]

        idx = .index_counter
        .store[idx] = {
            : artifact,
            : artifact_type,
            : time.time(),
            : time.time(),
            : 
        }

        .index_counter += 
         idx

     ():
        
         idx   .store:
             

        entry = .store[idx]
        entry[] = time.time()
        entry[] += 

         entry[]

     ():
        
         (.store.keys())