sage-rl-agent

name	sage-rl-agent
title	RL for Self-Improving Agent with Skill Library
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2512.17102
keywords	["reinforcement-learning","agents","skill-learning","self-improvement","grpo"]
description	Enable agents to continuously improve by accumulating reusable skills across sequential task chains. Train via GRPO across task sequences where skills persist and compound, provide dual rewards for both task completion and skill generation/reuse—improving completion rates 8.9% while reducing token costs by 59% compared to non-skill baselines.

Overview

SAGE (Skill Augmented GRPO for self-Evolution) addresses the challenge of agent self-improvement through skill accumulation. Rather than relearning solutions to repeated problems, agents build a persistent skill library and are rewarded for both using existing skills and generating new ones that prove useful in downstream tasks.

Core Technique

The key insight is that training across task sequences rather than individual tasks enables skill discovery and reuse.

Sequential Rollout Training: Instead of treating each task independently, train across chains of related tasks where skills accumulate.

# Sequential task training with skill persistence
class SAGE:
    def __init__(self):
        self.skill_library = {}
        self.task_chain = []

    def train_on_task_chain(self, task_sequence):
        """
        Process chain of similar tasks sequentially.
        Skills generated in early tasks are available for later tasks.
        """
        skill_library = {}  # Persistent across tasks

        for task_idx, task in enumerate(task_sequence):
            print(f"Task {task_idx}: {task.name}")

            # Rollout for this task (with available skills)
            trajectory = self.rollout_with_skills(task, skill_library)

            # Compute rewards (outcome + skill metrics)
            outcome_reward = compute_task_reward(trajectory)

            # Identify if agent generated new, useful skills
            new_skills = self.extract_skills(trajectory)
            for skill in new_skills:
                # Check if skill is useful for remaining tasks
                future_usefulness = assess_future_utility(
                    skill, task_sequence[task_idx+:]
                )
                 future_usefulness > threshold:
                    skill_library[skill.name] = skill
                    outcome_reward += skill_generation_bonus

            
            .update_via_grpo(trajectory, outcome_reward)

         skill_library