メニュー
Achieve emergent brevity in reasoning by retaining and up-weighting easy problems during RL training, implicitly regularizing solution length without explicit penalties while maintaining accuracy on hard problems.
デフォルトでは、最初にソースを確認する Prompt が選択されています。直接コマンドに切り替えるか、ローカルコピーをダウンロードすることもできます。
インストールを決める前に、SKILL.md と SkillsMP に表示されている付属ファイルをお読みください。
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
直接コマンドでは確認用 Prompt が省略されます。実行前にソースを確認してください。
npx skills add https://github.com/ADu2021/skillXiv --skill frugal-reasoning-short-math-solutionsコマンドは1行のまま表示されます。コピー前に横へスクロールして全体を確認してください。
ローカルで確認しますか？SkillsMP が現在取得できるファイルをダウンロードできます。
SKILL.md を表示中
SOC 職業分類に基づく
| name | frugal-reasoning-short-math-solutions |
| title | Shorter but not Worse: Frugal Reasoning via Easy Samples in Math RLVR |
| version | 0.0.2 |
| engine | skillxiv-v0.0.2-claude-opus-4.6 |
| license | MIT |
| url | https://arxiv.org/abs/2511.01937 |
| keywords | ["Efficient Reasoning","Length Control","Reinforcement Learning","Mathematical Problem Solving"] |
| description | Achieve emergent brevity in reasoning by retaining and up-weighting easy problems during RL training, implicitly regularizing solution length without explicit penalties while maintaining accuracy on hard problems. |
Standard RLVR filters out easy problems to focus on challenging ones. This creates a problem: models learn that harder problems require longer solutions, so they generate unnecessarily verbose reasoning even on solvable tasks. Frugal reasoning reverses this: by retaining moderately easy problems and up-weighting them, the model learns compact solution patterns. The result is "emergent brevity"—solutions nearly twice as short without explicit length penalties, while maintaining baseline accuracy.
The mechanism is implicit length regularization through training distribution bias.
Implicit Length Regularization via Easy Sample Up-Weighting:
1. Classify Problem Difficulty
Estimate which problems are easy vs. hard.
class ProblemDifficultyClassifier:
def __init__(self):
# Simple heuristics for difficulty estimation
self.difficulty_features = {
'num_digits': lambda x: x.count(str(i)) for i in range(),
: x: (x.count(op) op [, , , , ]),
: x: x.count(),
: x: (x.split())
}
():
features = {
: (problem_text.split()),
: (problem_text.count(op) op [, , , , ]),
: problem_text,
: problem_text
}
score = (
features[] / +
features[] / +
features[] * +
features[] *
)
difficulty = (, score / )
solution_length:
difficulty += (, solution_length / )
difficulty
():
difficulties = [.estimate_difficulty(p) p problems]
easy_threshold = np.percentile(difficulties, )
hard_threshold = np.percentile(difficulties, )
categorized = {
: [p p, d (problems, difficulties) d < easy_threshold],
: [p p, d (problems, difficulties) easy_threshold <= d < hard_threshold],
: [p p, d (problems, difficulties) d >= hard_threshold]
}
categorized
2. Implement Curriculum Weighting
Up-weight easy/medium problems in RL loss.
def compute_curriculum_weights(difficulty_scores, up_weighting=2.0):
"""Compute importance weights for curriculum"""
# Invert: easy (low difficulty) get higher weight
base_weights = 1.0 - np.array(difficulty_scores)
# Up-weight easy problems
curriculum_weights = np.where(
base_weights > 0.5, # Easy if difficulty < 0.5
base_weights * up_weighting, # Up-weight easy
base_weights # Normal weight for hard
)
# Normalize
curriculum_weights /= curriculum_weights.mean()
return curriculum_weights
def compute_grpo_loss_with_curriculum(
model_outputs, reference_outputs, rewards,
difficulty_scores, curriculum_upweight=2.0
):
"""GRPO loss weighted by curriculum"""
# Compute curriculum weights
weights = compute_curriculum_weights(difficulty_scores, curriculum_upweight)
# Group by reward quartiles
sorted_indices = np.argsort(rewards)
group_size = len(rewards) // 4
loss = 0
for quartile in range(4):
group_start = quartile * group_size
group_end = (quartile + 1) * group_size
group_indices = sorted_indices[group_start:group_end]
group_rewards = torch.tensor([rewards[i] for i in group_indices])
group_weights = torch.tensor([weights[i] for i in group_indices])
# Higher quartiles are positive examples
if quartile >= 2:
# Positive group: maximize likelihood
group_loss = -torch.log(torch.tensor([model_outputs[i] for i in group_indices]) + 1e-8)
else:
# Negative group: minimize likelihood
group_loss = torch.log(torch.tensor([reference_outputs[i] for i in group_indices]) + 1e-8)
# Weight by curriculum
weighted_loss = (group_loss * group_weights).mean()
loss += weighted_loss
return loss / 4
3. Implement Two-Stage Training
Stage 1: Learn brevity via easy samples. Stage 2: Improve on hard problems.
def two_stage_frugal_training(model, dataset, token_budget=16000):
"""Train with frugal curriculum"""
classifier = ProblemDifficultyClassifier()
# Categorize problems
categorized = classifier.categorize_problems([d['problem'] for d in dataset])
# Stage 1: Learn brevity with heavy easy-sample weighting
print("Stage 1: Learning brevity via easy samples...")
optimizer = torch.optim.Adam(model.parameters(), lr=1e-4)
stage1_data = categorized['easy'] + categorized['medium'] # Mix easy and medium
stage1_weights = compute_curriculum_weights(
[classifier.estimate_difficulty(p) for p in stage1_data],
up_weighting=3.0 # Heavy up-weighting for brevity
)
for epoch in range(3):
for problem, weight in zip(stage1_data, stage1_weights):
# Generate solution with token budget
solution = model.generate(problem, max_tokens=token_budget)
# Reward: correctness + brevity
correct = verify_solution(solution, problem)
length_efficiency = 1.0 / (1.0 + len(solution.split()) / 100)
reward = 0.8 * correct + 0.2 * length_efficiency
# Loss weighted by curriculum
loss = -torch.log(torch.tensor(reward)) * weight
optimizer.zero_grad()
loss.backward()
optimizer.step()
# Stage 2: Improve on hard problems with learned brevity
print("Stage 2: Improving on hard problems...")
stage2_data = categorized['hard']
stage2_weights = compute_curriculum_weights(
[classifier.estimate_difficulty(p) for p in stage2_data],
up_weighting=1.0 # Normal weighting for hard problems
)
for epoch in range(10):
for problem, weight in zip(stage2_data, stage2_weights):
# Generate with token budget (model has learned brevity)
solution = model.generate(problem, max_tokens=token_budget)
# Reward: emphasis on correctness
correct = verify_solution(solution, problem)
length_efficiency = 1.0 / (1.0 + len(solution.split()) / 100)
reward = 0.95 * correct + 0.05 * length_efficiency
loss = -torch.log(torch.tensor(reward)) * weight
optimizer.zero_grad()
loss.backward()
optimizer.step()
return model
4. Evaluate Emergent Brevity
Measure length and accuracy trade-off.
def evaluate_frugal_reasoning(model, test_problems, ground_truth_answers):
"""Evaluate model on length and accuracy"""
results = {
'accuracies': [],
'solution_lengths': [],
'tokens_per_problem': []
}
for problem, gt_answer in zip(test_problems, ground_truth_answers):
# Generate solution
solution = model.generate(problem, max_tokens=16000)
# Check correctness
predicted_answer = extract_answer(solution)
correct = predicted_answer == gt_answer
results['accuracies'].append(correct)
results['solution_lengths'].append(len(solution.split()))
results['tokens_per_problem'].append(len(solution.split()) * 1.3) # Token estimate
# Compute metrics
accuracy = np.mean(results['accuracies'])
avg_length = np.mean(results['solution_lengths'])
efficiency = np.mean(results['tokens_per_problem'])
print(f"Accuracy: {accuracy:.1%}")
print(f"Avg solution length: {avg_length:.0f} words")
print(f"Token efficiency: {efficiency:.0f} tokens/problem")
return results
When to Use:
Hyperparameters:
When NOT to Use:
Pitfalls:
Key Insight: The mechanism works because easy samples teach "fast solution patterns." Once the model learns to solve easy problems quickly, these patterns transfer to hard problems. No explicit length penalty needed.