skill-rl-recursive-distillation

name	skill-rl-recursive-distillation
title	SkillRL: Evolving Agents via Recursive Skill-Augmented RL
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2602.08234
keywords	["Skill Distillation","Agent Learning","Experience Reuse","Hierarchical Skills","Policy Improvement"]
description	Improve agent performance by autonomously distilling behavioral patterns from trajectories into reusable skills, then using these skills to guide future decisions. Achieves 89.9% success on ALFWorld through differential processing of success vs failure episodes and dynamic skill library evolution.

SkillRL: Recursive Skill-Augmented Reinforcement Learning

Agent performance plateaus when policies must rediscover past insights repeatedly. SkillRL addresses this by automatically extracting behavioral patterns from interaction history into compact, reusable skills that guide future decision-making. Rather than storing raw trajectories, the system distills strategic patterns and failure lessons, creating a skill library that grows with the agent.

Core Concept

SkillRL processes trajectories differentially:

• Success episodes → extract strategic patterns (10-20× compression vs raw trajectory)
• Failure episodes → extract failure lessons capturing what went wrong

Skills organize hierarchically: general skills (exploration, state management) and task-specific skills. During decision-making, the agent retrieves relevant skills via semantic similarity, reducing context overhead while maintaining reasoning quality.

The skill library evolves recursively: after validation epochs, failure modes generate new skills or refine existing ones, creating a virtuous cycle where improved policies encounter new challenges.

Architecture Overview

• Experience Processing: Separate success trajectories (extract patterns) and failure trajectories (extract lessons)
• Skill Library (SkillBank): Two-tier organization—general skills (universal) and task-specific skills
• Semantic Retrieval: Use embedding similarity to retrieve relevant skills for current state
• Dynamic Evolution: Analyze failure modes to generate new skills or update existing ones
• Integration: Skills prepend to context during policy rollouts

Implementation

Process trajectories into skills by extracting high-level patterns:

def extract_skills_from_trajectory(trajectory, success=True):
    """Extract skills from a trajectory (success or failure)."""
    if success:
        # For successful trajectories: extract strategic patterns
        # E.g., "When encountering X, use strategy Y"
        skill = {
            'type': 'strategic',
            'condition': identify_key_decision_points(trajectory),
            : abstract_action_sequence(trajectory),
            : 
        }
    :
        
        
        failure_point = identify_failure_point(trajectory)
        skill = {
            : ,
            : failure_point[],
            : failure_point[],
            : failure_point[],
            : 
        }
     skill

 ():
    
     skill[] == :
         
    :
         


skill_library = {: [], : {}}
 traj  trajectories:
     traj[]:
        skill = extract_skills_from_trajectory(traj, success=)
        skill_library[].append(compress_skill_text(skill))
    :
        skill = extract_skills_from_trajectory(traj, success=)
        task = traj[]
         task   skill_library[]:
            skill_library[][task] = []
        skill_library[][task].append(compress_skill_text(skill))