ruscarl-rubric-scaffolded-rl

name	ruscarl-rubric-scaffolded-rl
title	RuscaRL: Rubric-Scaffolded RL for General LLM Reasoning
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2508.16949
keywords	["reinforcement-learning","rubric-guidance","exploration-guidance","reasoning-scaffolding","curriculum-learning"]
description	Guide LLM exploration through rubric-based scaffolding that gradually diminishes, enabling models to internalize reasoning patterns while maintaining exploration quality for robust RL training.

RuscaRL: Rubric-Scaffolded Reinforcement Learning

Core Concept

The core challenge in LLM RL is exploration: what cannot be explored cannot be learned. RuscaRL (Rubric-Scaffolded RL) addresses this by using checklist-style rubrics in two phases: first as explicit guidance during exploration (gradually fading), then as reference for scoring. This approach breaks the constraint that LLM limitations restrict sample quality, enabling models to learn general reasoning capabilities through guided exploration.

Architecture Overview

• Exploration Rubrics: Checklist guidance for high-quality response generation
• Gradual Scaffolding Fade: Curriculum learning removing explicit guidance
• Rubric-Based Scoring: Reference for reward computation
• LLM-as-Judge: Automated evaluation using rubrics
• General Reasoning Transfer: Patterns learned with rubrics transfer without them

Implementation Steps

1. Design Task-Specific Rubrics

Create structured evaluation checklist templates:

from dataclasses import dataclass
from typing import List, Dict, Any
from enum import Enum

class RubricLevel(Enum):
    EXPLICIT = "explicit"      # Full rubric in prompt
    IMPLICIT = "implicit"      # Rubric referenced but not detailed
    ABSENT = "absent"           # No rubric mentioned

@dataclass
class RubricCriterion:
    """Single rubric evaluation criterion."""
    name: str
    description: str
    exemplar_good: str
    exemplar_bad: str
    weight: float = 


 :
    
    task_name: 
    criteria: [RubricCriterion]
    instructions: 

     () -> :
        
        guidance = 
        guidance += 

         criterion  .criteria:
            guidance += 
            guidance += 
            guidance += 

         guidance

 :
    

     ():
        .rubrics: [, TaskRubric] = {}

     ():
        
        .rubrics[task_type] = rubric

     () -> TaskRubric:
         .rubrics.get(task_type)

     () -> TaskRubric:
        
         TaskRubric(
            task_name=,
            criteria=[
                RubricCriterion(
                    name=,
                    description=,
                    exemplar_good=,
                    exemplar_bad=,
                    weight=
                ),
                RubricCriterion(
                    name=,
                    description=,
                    exemplar_good=,
                    exemplar_bad=,
                    weight=
                ),
                RubricCriterion(
                    name=,
                    description=,
                    exemplar_good=,
                    exemplar_bad=,
                    weight=
                ),
                RubricCriterion(
                    name=,
                    description=,
                    exemplar_good=,
                    exemplar_bad=,
                    weight=
                )
            ],
            instructions=
        )