experiential-reinforcement-learning

name	experiential-reinforcement-learning
title	Experiential Reinforcement Learning
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2602.13949
keywords	["Reinforcement Learning","Reflection","Revision","Behavioral Distillation","Feedback Integration"]
description	Improve RL training efficiency by embedding explicit reflection and revision loops. Models generate initial responses, receive feedback, produce self-reflections describing improvements, revise their attempts, and distill successful corrections into the base policy. Achieves up to 81% improvement on complex tasks through structured behavioral change.

Experiential Reinforcement Learning

Problem Context

Standard RL trains models to implicitly discover how failures should translate into behavioral change through undirected trial-and-error. This is inefficient and wastes signal from corrective feedback. Experiential RL makes the learning process explicit: models generate reflections on failures, attempt revisions based on feedback, and internalize successful corrections. This transforms reactive behavior optimization into structured experience learning.

Core Concept

ERL operates in three phases: (1) initial attempt on a task, (2) reflection and revision (model generates explanation of improvements and attempts again), (3) internalization via distillation. A cross-episode reflection memory stores successful corrective patterns discovered during training, enabling reuse across tasks.

Architecture Overview

• Initial attempt: Generate first response to task
• Reflection generation: Produce reasoning about improvements given feedback
• Revision step: Attempt task again using reflection insights
• Success detection: Identify when revision succeeds
• Distillation: Consolidate successful revisions into base policy
• Reflection memory: Store and retrieve successful correction patterns

Implementation

Step 1: Task attempt and feedback collection

import torch
from typing import Dict, List, Tuple
from collections import defaultdict

class TaskAttempt:
    """Single task attempt with response and feedback."""

    def __init__(self, task: str, initial_response: str):
        self.task = task
        self.initial_response = initial_response
        self.feedback = None
        self.revision = 
        .revision_success = 

     ():
        
        .feedback = feedback
        .initial_success = success


 :
    

     ():
        .max_memory = max_memory
        .reflections = defaultdict()  
        .corrections = []  

     ():
        
        .reflections[task_type].append({
            : reflection,
            : initial_response,
            : revised_response,
            : task_description
        })

        .corrections.append((initial_response, reflection, revised_response))

        
         (.corrections) > .max_memory:
            .corrections = .corrections[-.max_memory:]

     () -> []:
        
         task_type   .reflections:
             []

        candidates = .reflections[task_type]

         query_embedding  :
            
             candidates[-top_k:]

        
        
         candidates[-top_k:]