memory-as-action

name	memory-as-action
title	Memory as Action: Autonomous Context Curation for Long-Horizon Agentic Tasks
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2510.12635
keywords	["memory-management","context-curation","long-horizon","reinforcement-learning","context-efficiency"]
description	Treat memory management as learnable RL policy actions (delete/insert) rather than fixed mechanisms. Enable models to autonomously decide what to keep, remove, or add to context, reducing average context length by 51% while matching larger models.

Memory as Action: Learning to Manage Context Efficiently

Long-context models suffer from attention dilution as context grows. Rather than using fixed memory management rules, Memory as Action lets the model learn what information to retain through reinforcement learning, optimizing memory operations as policy actions.

Core insight: working memory should be dynamic and task-aware. By treating memory deletion and insertion as learnable actions, models learn to maintain only decision-critical context, reducing context length by 51% while improving accuracy through better attention focus.

Core Concept

Memory as Policy Action: Define memory operations (delete, insert) as learnable actions optimized through RL, not static rules. The model decides what to keep based on reasoning state.

In-Place Editing: Rather than regenerating context, make surgical edits: delete irrelevant information, insert new conclusions. This is interpretable and efficient.

End-to-End Optimization: Joint optimization of context and task performance ensures memory serves the actual reasoning process.

Architecture Overview

• Memory State Tracker: Maintains current context window
• Action Policy: Learns which tokens to delete/insert
• Task Executor: Uses curated memory for reasoning
• Reward Signal: Task performance + memory efficiency

Implementation Steps

Stage 1: Define Memory Operations

Implement memory editing as differentiable operations:

import torch
import torch.nn as nn

class MemoryEditor(nn.Module):
    def __init__(self, hidden_dim=768, max_memory_size=2048):
        super().__init__()

        self.hidden_dim = hidden_dim
        self.max_memory_size = max_memory_size

        # Policy for deciding what to delete
        self.delete_policy = nn.Sequential(
            nn.Linear(hidden_dim * 2, hidden_dim),
            nn.ReLU(),
            nn.Linear(hidden_dim, ),
            nn.Sigmoid()
        )

        
        .insert_policy = nn.Sequential(
            nn.Linear(hidden_dim, hidden_dim),
            nn.ReLU(),
            nn.Linear(hidden_dim, hidden_dim)
        )

     ():
        

        batch_size, seq_len, _ = memory_tokens.shape

        
        current_expanded = current_state.unsqueeze().expand(
            -, seq_len, -
        )

        
        combined = torch.cat(
            [memory_tokens, current_expanded],
            dim=-
        )

        
        delete_scores = .delete_policy(combined).squeeze(-)

         delete_scores  

     ():
        

        insertion_vectors = .insert_policy(current_state)

         insertion_vectors  

     ():
        

        batch_size, seq_len, dim = memory_tokens.shape

        
        delete_mask = (delete_scores > delete_threshold)

        
        kept_tokens = memory_tokens[~delete_mask]

        
        new_size = kept_tokens.shape[]

        
         new_size +  <= .max_memory_size:
            new_memory = torch.cat(
                [kept_tokens, insertion_vectors.unsqueeze()],
                dim=
            )
        :
            new_memory = kept_tokens[:.max_memory_size]

         new_memory, {
            : delete_mask.(),
            :   (new_size + ) <= .max_memory_size  ,
            : new_memory.shape[]
        }