frugal-reasoning-short-math-solutions

name	frugal-reasoning-short-math-solutions
title	Shorter but not Worse: Frugal Reasoning via Easy Samples in Math RLVR
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2511.01937
keywords	["Efficient Reasoning","Length Control","Reinforcement Learning","Mathematical Problem Solving"]
description	Achieve emergent brevity in reasoning by retaining and up-weighting easy problems during RL training, implicitly regularizing solution length without explicit penalties while maintaining accuracy on hard problems.

Title: Train Efficient Reasoners by Including Easy Problems During RL

Standard RLVR filters out easy problems to focus on challenging ones. This creates a problem: models learn that harder problems require longer solutions, so they generate unnecessarily verbose reasoning even on solvable tasks. Frugal reasoning reverses this: by retaining moderately easy problems and up-weighting them, the model learns compact solution patterns. The result is "emergent brevity"—solutions nearly twice as short without explicit length penalties, while maintaining baseline accuracy.

The mechanism is implicit length regularization through training distribution bias.

Core Concept

Implicit Length Regularization via Easy Sample Up-Weighting:

• Data Curriculum: Include easy, medium, and hard problems (don't filter easy)
• Weighting Strategy: Up-weight easy/medium problems in RL loss
• Emergent Brevity: Model learns efficiency naturally without explicit length penalties
• Two-Stage Training: Stage 1 learns brevity, Stage 2 improves on harder problems
• Token Budget: Maintain fixed context limit (16K tokens typical)

Architecture Overview

• Problem Difficulty Estimation: Automatic or heuristic classification
• Curriculum Weighting: Exponential down-weighting of hard problems
• GRPO Optimization: Group Relative Policy Optimization on mixed-difficulty data
• Token Budget Constraint: Enforce maximum sequence length
• Evaluation: Accuracy on hard problems + length efficiency

Implementation Steps

1. Classify Problem Difficulty

Estimate which problems are easy vs. hard.

class ProblemDifficultyClassifier:
    def __init__(self):
        # Simple heuristics for difficulty estimation
        self.difficulty_features = {
            'num_digits': lambda x: x.count(str(i)) for i in range(),
            :  x: (x.count(op)  op  [, , , , ]),
            :  x: x.count(),
            :  x: (x.split())
        }

     ():
        
        features = {
            : (problem_text.split()),
            : (problem_text.count(op)  op  [, , , , ]),
            :   problem_text,
            :   problem_text
        }

        
        score = (
            features[] /  +
            features[] /  +
            features[] *  +
            features[] * 
        )

        
        difficulty = (, score / )

        
         solution_length:
            
            difficulty += (, solution_length / )

         difficulty

     ():
        
        difficulties = [.estimate_difficulty(p)  p  problems]

        easy_threshold = np.percentile(difficulties, )
        hard_threshold = np.percentile(difficulties, )

        categorized = {
            : [p  p, d  (problems, difficulties)  d < easy_threshold],
            : [p  p, d  (problems, difficulties)  easy_threshold <= d < hard_threshold],
            : [p  p, d  (problems, difficulties)  d >= hard_threshold]
        }

         categorized