Skip to main content Inicio Creadores adu2021 skillxiv memskill-evolving-memory-operations
memskill-evolving-memory-operations Build agents that evolve their own memory operations by learning a skill bank of memory transformations and periodically discovering new skills from challenging cases, enabling adaptive memory management that improves with scale.
Ir a la instalación Skills Marketplace Descubre y explora habilidades de IA creadas por la comunidad.
Instalar con Codex o Claude Copia este prompt, pégalo en Codex, Claude u otro asistente, y deja que revise la página de la skill y la instale por ti.
Mostrar detalles del prompt Un comando directo omite el prompt de revisión. Revisa el origen antes de ejecutarlo.
npx skills add https://github.com/ADu2021/skillXiv --skill memskill-evolving-memory-operationsEl comando permanece en una sola línea. Desplázate horizontalmente para revisarlo antes de copiarlo.
¿Prefieres una copia local? Descarga los archivos que SkillsMP tiene disponibles ahora.
Descargar Zip Descargando... Más de este repositorio meaningful-kebab-case-name Convert arXiv papers into ready-to-use agent skills using category-aware extraction. First classifies the paper into one or more of 11 research categories, then applies a specialized extraction pipeline for each category — because different types of papers produce different types of usable knowledge. A single paper can yield multiple skills if it spans categories. Use this skill whenever the user wants to turn a paper into a skill, extract practical techniques from research, build a skill library from papers, convert arXiv papers into reusable agent instructions, or batch-process multiple papers into skills. Also trigger when someone asks about extracting actionable knowledge from papers, making research practical for LLM agents, or systematically converting academic contributions into structured agent capabilities.
action-quantization-behavior-cloning Establish regret bounds for behavior cloning with discretized actions combining statistical error and quantization error terms. Prove smoothness requirements for safe quantizer design, show that learning-based quantizers fail these requirements, and propose model-based augmentation to reduce error dependence from H² to H.
adaptive-lora-personalized-ranks Dynamically allocate LoRA ranks per-layer during fine-tuning instead of using fixed uniform ranks. Learn optimal rank for each layer and subject via variational framework with discretized exponential distribution, reducing memory footprint while maintaining fidelity and text-alignment.
Ocupaciones relacionadas SOC
Basado en la clasificación ocupacional SOC
name memskill-evolving-memory-operations title MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents version 0.0.2 engine skillxiv-v0.0.2-claude-opus-4.6 license MIT url https://arxiv.org/abs/2602.02474 keywords ["Agent Memory","Skill Learning","Self-Improvement","LLM Controller","Reinforcement Learning"] description Build agents that evolve their own memory operations by learning a skill bank of memory transformations and periodically discovering new skills from challenging cases, enabling adaptive memory management that improves with scale.
MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents
Problem Context
Existing LLM agent memory systems rely on static, hand-designed operations (insert, update, delete) that hard-code human assumptions. These fixed procedures fail when interaction patterns vary, histories grow longer, or task structure changes. Agents cannot adapt their memory strategies or learn what truly matters for their specific workflow.
Core Concept
MemSkill introduces [learnable skill banks, controller-executor architecture, LLM-guided evolution] to enable agents to discover and refine memory operations autonomously. A learned policy (controller) selects which skills to apply per context, while an LLM-based designer periodically identifies missing behaviors and proposes new skills through mining hard cases.
Architecture Overview
Skill bank : Learnable memory operations (Insert, Update, Delete, Skip) stored as prompts; initially hand-designedController : Neural policy selecting relevant skills for each context span (trained with PPO)Executor : LLM-based component applying selected skills to generate memoriesDesigner : LLM mining hard cases to identify skill gaps and propose new operationsTraining loop : Alternate between RL-based skill selection and skill bank evolution
Implementation
Step 1: Define initial skill bank
Create a basic set of learnable memory operations. Each skill is represented as a prompt specifying when and how to transform interaction traces.
class MemorySkillBank :
def __init__ (self ):
self .skills = {
'insert' : {
'description' : 'Add new information to memory' ,
'prompt' : 'Given the interaction: {interaction}, '
'extract and add the most important fact to memory.' ,
'parameters' : {}
},
'update' : {
'description' : 'Modify existing memory entries'
'prompt'
'Given the interaction: {interaction} and '
'current memory: {memory}, update the memory '
'if new information contradicts or extends existing facts.'
'parameters'
'delete'
'description'
'Remove outdated or irrelevant memories'
'prompt'
'Given the interaction: {interaction} and '
'current memory: {memory}, remove memories that '
'are no longer relevant or are contradicted.'
'parameters'
'skip'
'description'
'Do not modify memory'
'prompt'
'No memory modification needed.'
'parameters'
self
list
self
def
get_skill_prompt
self, skill_name, interaction, memory
"""Get the prompt for a specific skill."""
self
return
'prompt'
format
def
add_skill
self, skill_name, description, prompt
"""Add a new skill to the bank (called by Designer)."""
self
'description'
'prompt'
'parameters'
self
Step 2: Implement controller for skill selection Train a neural network to select which skills are relevant for each interaction context. Use PPO with task performance as reward.
class MemorySkillController :
def __init__ (self, input_dim=768 , num_skills=4 ):
self .encoder = nn.LSTM(input_dim, 256 , batch_first=True )
self .policy_head = nn.Linear(256 , num_skills)
self .value_head = nn.Linear(256 , 1 )
def select_skills (self, interaction_embedding, memory_embedding ):
"""
Select skills based on interaction and memory context.
Returns skill probabilities and sampled skill indices.
"""
combined = torch.cat([interaction_embedding, memory_embedding], dim=-1 )
encoded, _ = self .encoder(combined)
logits = self .policy_head(encoded[:, -1 , :])
probs = F.softmax(logits, dim=-1 )
value = self .value_head(encoded[:, -1 , :])
skill_mask = torch.multinomial(probs, num_samples=1 , replacement=False )
return probs, skill_mask, value
Step 3: Implement executor for applying skills Use an LLM to execute selected skills on interaction traces. Apply skills in sequence to transform memories.
class MemoryExecutor :
def __init__ (self, llm_client, skill_bank ):
self .llm = llm_client
self .skill_bank = skill_bank
def execute_skills (self, interaction, memory, skill_indices ):
"""
Apply selected skills to update memory.
"""
current_memory = memory
execution_trace = []
for skill_idx in skill_indices:
skill_name = self .skill_bank.skill_names[skill_idx]
skill_prompt = self .skill_bank.get_skill_prompt(
skill_name, interaction, current_memory
)
response = self .llm.generate(
prompt=skill_prompt,
max_tokens=200 ,
temperature=0.7
)
execution_trace.append({
'skill' : skill_name,
'input_memory' : current_memory,
'output' : response
})
current_memory = response
return current_memory, execution_trace
Step 4: Train controller with PPO Optimize the controller to select skills that maximize task performance. Use task reward signals to guide skill selection.
def train_controller_ppo (
controller, executor, skill_bank, trajectories,
task_reward_fn, optimizer, num_epochs=3
):
"""
Train skill selection controller with PPO on task rewards.
"""
for epoch in range (num_epochs):
total_loss = 0.0
for trajectory in trajectories:
interaction_seq = trajectory['interactions' ]
memory_seq = trajectory['memories' ]
task_reward = task_reward_fn(trajectory)
probs_seq = []
skill_masks = []
values = []
for i, (interaction, memory) in enumerate (
zip (interaction_seq, memory_seq)
):
inter_emb = embed_interaction(interaction)
mem_emb = embed_memory(memory)
probs, skill_mask, value = controller.select_skills(
inter_emb, mem_emb
)
probs_seq.append(probs)
skill_masks.append(skill_mask)
values.append(value)
returns = compute_returns([task_reward] * len (values), gamma=0.99 )
advantages = returns - torch.tensor(values)
old_log_probs = torch.log(torch.gather(
torch.stack(probs_seq), 1 , torch.stack(skill_masks)
)).detach()
new_probs = torch.stack(probs_seq)
new_log_probs = torch.log(torch.gather(new_probs, 1 , torch.stack(skill_masks)))
ratio = torch.exp(new_log_probs - old_log_probs)
surr1 = ratio * advantages
surr2 = torch.clamp(ratio, 0.95 , 1.05 ) * advantages
policy_loss = -torch.min (surr1, surr2).mean()
value_loss = F.smooth_l1_loss(torch.cat(values), returns)
loss = policy_loss + 0.5 * value_loss
optimizer.zero_grad()
loss.backward()
optimizer.step()
total_loss += loss.item()
print (f"Epoch {epoch} : Loss = {total_loss / len (trajectories)} " )
Step 5: Implement designer for skill discovery Mine challenging cases where the current skill set fails. Use LLM to propose new skills addressing gaps.
class MemorySkillDesigner :
def __init__ (self, llm_client ):
self .llm = llm_client
def mine_hard_cases (self, trajectories, success_threshold=0.5 ):
"""
Identify trajectories where memory management failed.
"""
hard_cases = []
for trajectory in trajectories:
final_reward = trajectory.get('reward' , 0 )
if final_reward < success_threshold:
hard_cases.append({
'interactions' : trajectory['interactions' ],
'memory_operations' : trajectory['skills_applied' ],
'outcome' : trajectory.get('failure_reason' , '' ),
'reward' : final_reward
})
return hard_cases
def propose_new_skill (self, hard_case, existing_skills ):
"""
Use LLM to propose a new skill addressing a failure case.
"""
case_desc = f"""
Interaction sequence: {hard_case['interactions' ]}
Applied skills: {hard_case['memory_operations' ]}
Failure: {hard_case['outcome' ]}
Existing skills: {list (existing_skills.keys())}
"""
prompt = f"""
Given this memory management failure case:
{case_desc}
Propose a new memory operation skill that would address this failure.
Provide:
1. Skill name (kebab-case)
2. Description of what it does
3. Prompt template for LLM execution
Format:
Name: [skill-name]
Description: [description]
Prompt: [prompt template with {{interaction}} and {{memory}} placeholders]
"""
response = self .llm.generate(
prompt=prompt,
max_tokens=300 ,
temperature=0.7
)
new_skill = parse_skill_proposal(response)
return new_skill
def evolve_skill_bank (self, skill_bank, hard_cases, top_k=3 ):
"""
Discover and add top-k new skills to the skill bank.
"""
proposed_skills = []
for hard_case in hard_cases[:top_k]:
new_skill = self .propose_new_skill(hard_case, skill_bank.skills)
proposed_skills.append(new_skill)
for skill in proposed_skills:
skill_bank.add_skill(
skill['name' ],
skill['description' ],
skill['prompt' ]
)
return proposed_skills
Step 6: Main training loop alternating controller and designer Implement the core loop: train controller, mine hard cases, evolve skills, repeat.
def train_memskill_agent (
skill_bank, controller, executor, designer,
all_trajectories, task_reward_fn,
num_iterations=10 , hard_case_threshold=0.5
):
"""
Main loop: train controller → mine failures → evolve skills → repeat.
"""
controller_optimizer = torch.optim.Adam(controller.parameters(), lr=1e-4 )
for iteration in range (num_iterations):
print (f"Iteration {iteration + 1 } " )
print (" Training controller with PPO..." )
train_controller_ppo(
controller, executor, skill_bank,
all_trajectories, task_reward_fn,
controller_optimizer
)
print (" Mining hard cases..." )
hard_cases = designer.mine_hard_cases(
all_trajectories,
success_threshold=hard_case_threshold
)
if len (hard_cases) > 0 :
print (f" Proposing new skills ({len (hard_cases)} failures)..." )
new_skills = designer.evolve_skill_bank(
skill_bank, hard_cases, top_k=3
)
print (f" Added {len (new_skills)} new skills" )
eval_reward = evaluate_agent(
controller, executor, skill_bank, all_trajectories
)
print (f" Evaluation reward: {eval_reward} " )
Practical Guidance When to use : Long-horizon agent tasks (conversational, embodied control) where memory management is critical and task patterns vary. Less beneficial for one-off reasoning or static retrieval scenarios.
Controller hidden size (256-512): larger for more complex contexts
PPO learning rate (1e-4 to 5e-4): conservative to avoid instability
Skill mining threshold (0.3-0.7): lower catches more failure cases
Evolution frequency: every 10-20 iterations if using growing trajectory data
Skill bank explosion: periodically prune low-utility skills by measuring frequency
Designer over-proposes: filter new skills by simulating on hard cases first
Controller convergence: use gradient clipping and value normalization for stability
Memory bloat: cap memory size; implement retention policies (recency, importance)
Scaling : Skill bank grows linearly with iterations (add 2-3 skills per round). Controller training cost is linear in trajectory count. Recommend evaluating every 5-10 iterations to track improvement.
Reference Paper: https://arxiv.org/abs/2602.02474
Code: Available at author's repository
Related work: Memory-augmented neural networks, meta-learning, agent architecture design
