tarot-test-driven-curriculum-rl

name	tarot-test-driven-curriculum-rl
title	TAROT: Test-driven Curriculum RL for Code Generation
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2602.15449
keywords	["curriculum learning","reinforcement learning","code generation","test-driven development","model capability adaptation"]
description	Design capability-adaptive curricula for code generation by constructing per-problem test suites (basic, intermediate, complex, edge cases) and dynamically weighting training signals based on model capability rather than problem difficulty alone. Optimal curricula vary by model capacity: weaker models benefit from easy-to-hard progression while stronger models learn faster with complex-first strategies, enabling 1 dataset to serve multiple model scales efficiently.

TAROT: Capability-Adaptive Test-Driven Curriculum for Code RL

Training reinforcement learning policies for code generation faces a fundamental trade-off: complex problems provide rich learning signal but can overwhelm models during early training, while simpler problems are easily solved but offer limited learning benefit. Traditional curriculum approaches sort problems by difficulty and train sequentially, assuming optimal progression is universal. This ignores a critical insight: optimal curriculum structure depends on model capability, not just problem count or parameter density.

The challenge is discovering which difficulty progression maximizes stable competency acquisition for a specific model at a specific stage of training. Applying the same curriculum to models of different scales or training phases often leads to either early saturation or optimization instability.

Core Concept

TAROT implements capability-adaptive curriculum learning by decomposing each coding problem into an internal test hierarchy rather than treating each problem atomically. For each problem, the framework constructs a four-tier test suite (basic, intermediate, complex, edge-case) that mirrors test-driven development practices. During training, a capability-adaptive allocator determines what fraction of training compute each tier receives, and a reward weighting mechanism scales contributions based on the model's current capability level.

The key innovation is decoupling curriculum progression from raw problem difficulty by using two learnable components: a curriculum allocator (proportion of training focus per tier) and tier-specific reward weights (priority per tier based on observed capability).

Architecture Overview

• Test Suite Construction: For each code problem, generate tests at 4 difficulty levels (basic validation through edge cases)
• Curriculum Allocator: Specifies target proportion of training trajectories per tier (e.g., 30% basic, 40% intermediate, 20% complex, 10% edge)
• Capability Scorer: Monitor model success rate on each tier to infer current capability level
• Reward Weighting: Dynamically scale per-tier rewards such that capability-appropriate successes contribute more to the training objective
• Progression Controller: Shift allocator weights over training epochs based on capability trajectory

Implementation

Construct the test suite for a single problem by generating tests at each difficulty level, then implement the capability-adaptive weighting during reward computation:

def construct_test_suite(problem_description, num_tests_per_tier=):
    
    tiers = {
        : ,
        : ,
        : ,
        : 
    }

    test_suite = {}
     tier_name, tier_description  tiers.items():
        
        test_suite[tier_name] = generate_tests(
            problem_description, tier_description, num_tests_per_tier
        )
     test_suite