opensir-self-play-reasoning

name	opensir-self-play-reasoning
title	OpenSIR: Open-Ended Self-Improving Reasoner
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2511.00602
keywords	["Self-Play","Autonomous Learning","Mathematical Reasoning","Curriculum Learning","Problem Generation"]
description	Enable open-ended mathematical learning through co-evolutionary teacher-student self-play where a single model alternates roles to generate progressively harder problems and solve them, creating a virtuous cycle without external data.

Title: Bootstrap Mathematical Reasoning From Seed Problems Without Annotation

Traditional supervised learning requires massive annotated datasets. OpenSIR achieves autonomous improvement through co-evolutionary self-play: a single model acts as both problem-setter (teacher) and problem-solver (student), generating novel problems calibrated to current capability and solving them. The framework requires only a small seed of initial problems, then bootstraps to arbitrary complexity.

The key is maintaining optimal difficulty: problems easy enough to learn from but hard enough to improve capability.

Core Concept

Co-Evolutionary Teacher-Student Self-Play:

• Single Policy, Two Roles: Model alternates between generating problems (teacher) and solving them (student)
• Difficulty Calibration: Teacher learns to generate problems at ~70% solve rate (optimal difficulty)
• Diversity Rewards: Encourage distinct mathematical concepts vs. repeating same patterns
• No External Supervision: Learn entirely from self-generated data
• Emergent Curriculum: Naturally progress from arithmetic to advanced mathematics

Architecture Overview

• Problem Generator (Teacher): Produces novel math problems conditioned on difficulty level and concept diversity
• Problem Solver (Student): Generates step-by-step solutions via reasoning
• Capability Estimator: Tracks model performance across problem categories
• Curriculum Controller: Adjusts teacher difficulty based on student performance
• Scoring System: Rewards for both solution accuracy and problem novelty

Implementation Steps

1. Implement Teacher (Problem Generation)

Design the teacher to generate diverse, well-formed mathematical problems.

class MathTeacher(nn.Module):
    def __init__(self, model, difficulty_levels=[1, 2, 3, 4, 5]):
        self.model = model
        self.difficulty_levels = difficulty_levels
        .concept_tracker = defaultdict()

     ():
         difficulty  :
            difficulty = np.random.choice(.difficulty_levels)

        
        prompt = 

        problem = .model.generate(prompt, max_tokens=)

        
         concepts:
            .concept_tracker[concepts] += 

         problem, difficulty

     ():
        
        problems = []
         _  (num_problems):
            
            difficulty = np.random.choice(.difficulty_levels)
            problem, _ = .generate_problem(difficulty=difficulty)
            problems.append(problem)
         problems