learning-on-the-job-test-time-curricula

name	learning-on-the-job-test-time-curricula
title	Learning on the Job: Test-Time Curricula for Targeted Reinforcement Learning
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2510.04786
keywords	["test-time learning","curriculum learning","continual training","task-specific adaptation"]
description	Enable models to autonomously improve on target tasks during inference via test-time curricula (TTC-RL). Automatically select task-relevant training examples and apply RL to continue learning. Achieve 1.8x improvement on AIME25 math benchmarks and 2.1x on CodeElo competitive coding by learning task-specific skills at test time without human curation.

Learning on the Job: Test-Time Curricula for Targeted RL

Core Concept

Traditional training assumes the task distribution is fixed. Learning on the Job (LotJ) flips this assumption: during inference on a target task, the model autonomously assembles a task-specific curriculum from available training data and continues learning via RL. This enables dramatic improvements (1.8-2.1x) on challenging benchmarks by specializing to each problem's characteristics.

Architecture Overview

• Test-Time Curriculum Assembly: Automatically select relevant training examples from large pools without human curation
• Task-Specific Skill Development: Apply RL to continue training on examples most relevant to target task
• Continual Inference-Time Learning: Extend test-time scaling paradigm beyond planning to actual policy updates
• Multi-Domain Generalization: Works across mathematical reasoning (AIME25), competitive coding (CodeElo), diverse task types
• Stateful Learning: Maintain learned skills across examples within same inference session

Implementation Steps

1. Test-Time Curriculum Assembly

Automatically select which training examples to study for each target task.

class TestTimeCurriculumAssembler:
    def __init__(self, training_pool, embedding_model='gpt-4.1'):
        self.training_pool = training_pool  # All available training examples
        self.embedder = embedding_model
        self.selected_curriculum = []

    def assemble_curriculum(self, target_task, curriculum_size=100, budget=50):
        """
        Assemble task-specific curriculum from training pool.

        Args:
            target_task: Target problem to solve
            curriculum_size: Max examples in curriculum
            budget: RL training steps available
        """

        # Step 1: Embed target task
        target_embedding = self.embedder.embed(target_task)

        
        candidate_examples = []
         example  .training_pool:
            example_embedding = .embedder.embed(example[])
            similarity = cosine_similarity(target_embedding, example_embedding)
            candidate_examples.append((similarity, example))

        
        candidate_examples.sort(reverse=)

        
        
        selected = []
        selected_solutions = ()

         similarity, example  candidate_examples[:curriculum_size * ]:
            solution_pattern = ._extract_pattern(example[])

             solution_pattern   selected_solutions:
                selected.append(example)
                selected_solutions.add(solution_pattern)

                 (selected) >= curriculum_size:
                    

        
        .selected_curriculum = ._order_by_difficulty(selected, target_task)

         .selected_curriculum

     ():
        
        
        keywords = [, , , , ]
        patterns = [kw  kw  keywords  kw  solution.lower()]
         (patterns)  patterns  (,)

     ():
        
        difficulty_scores = []

         example  examples:
            
            difficulty = (example[].split()) /   
            similarity_to_target = cosine_similarity(
                .embedder.embed(example[]),
                .embedder.embed(target_task)
            )
            
            score = difficulty * ( - similarity_to_target)

            difficulty_scores.append((score, example))

        difficulty_scores.sort()
         [ex  _, ex  difficulty_scores]