菜单
Achieve emergent brevity in reasoning by retaining and up-weighting easy problems during RL training, implicitly regularizing solution length without explicit penalties while maintaining accuracy on hard problems.
默认使用会先检查来源的 Prompt；你也可以切换为直接命令，或下载本地副本。
决定是否安装前，请先阅读 SKILL.md，以及 SkillsMP 当前展示的配套文件。
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
直接命令不会经过审查 Prompt；运行前请先检查来源。
npx skills add https://github.com/ADu2021/skillXiv --skill frugal-reasoning-short-math-solutions命令会保持在同一行。复制前请横向滚动并检查完整内容。
想先保存到本地？可下载 SkillsMP 当前能够提供的文件。
基于 SOC 职业分类
正在显示 SKILL.md
| name | frugal-reasoning-short-math-solutions |
| title | Shorter but not Worse: Frugal Reasoning via Easy Samples in Math RLVR |
| version | 0.0.2 |
| engine | skillxiv-v0.0.2-claude-opus-4.6 |
| license | MIT |
| url | https://arxiv.org/abs/2511.01937 |
| keywords | ["Efficient Reasoning","Length Control","Reinforcement Learning","Mathematical Problem Solving"] |
| description | Achieve emergent brevity in reasoning by retaining and up-weighting easy problems during RL training, implicitly regularizing solution length without explicit penalties while maintaining accuracy on hard problems. |
Standard RLVR filters out easy problems to focus on challenging ones. This creates a problem: models learn that harder problems require longer solutions, so they generate unnecessarily verbose reasoning even on solvable tasks. Frugal reasoning reverses this: by retaining moderately easy problems and up-weighting them, the model learns compact solution patterns. The result is "emergent brevity"—solutions nearly twice as short without explicit length penalties, while maintaining baseline accuracy.
The mechanism is implicit length regularization through training distribution bias.
Implicit Length Regularization via Easy Sample Up-Weighting:
1. Classify Problem Difficulty
Estimate which problems are easy vs. hard.
class ProblemDifficultyClassifier:
def __init__(self):
# Simple heuristics for difficulty estimation
self.difficulty_features = {
'num_digits': lambda x: x.count(str(i)) for i in range(),
: x: (x.count(op) op [, , , , ]),
: x: x.count(),
: x: (x.split())
}
():
features = {
: (problem_text.split()),
: (problem_text.count(op) op [, , , , ]),
: problem_text,
: problem_text
}
score = (
features[] / +
features[] / +
features[] * +
features[] *
)
difficulty = (, score / )
solution_length:
difficulty += (, solution_length / )
difficulty
():
difficulties = [.estimate_difficulty(p) p problems]
easy_threshold = np.percentile(difficulties, )
hard_threshold = np.percentile(difficulties, )
categorized = {
: [p p, d (problems, difficulties) d < easy_threshold],
: [p p, d (problems, difficulties) easy_threshold <= d < hard_threshold],
: [p p, d (problems, difficulties) d >= hard_threshold]
}
categorized
2. Implement Curriculum Weighting
Up-weight easy/medium problems in RL loss.
def compute_curriculum_weights(difficulty_scores, up_weighting=2.0):
"""Compute importance weights for curriculum"""
# Invert: easy (low difficulty) get higher weight
base_weights = 1.0 - np.array(difficulty_scores)
# Up-weight easy problems
curriculum_weights = np.where(
base_weights > 0.5, # Easy if difficulty < 0.5
base_weights * up_weighting, # Up-weight easy
base_weights # Normal weight for hard
)
# Normalize
curriculum_weights /= curriculum_weights.mean()
return curriculum_weights
def compute_grpo_loss_with_curriculum(
model_outputs, reference_outputs, rewards,
difficulty_scores, curriculum_upweight=2.0
):
"""GRPO loss weighted by curriculum"""
# Compute curriculum weights
weights = compute_curriculum_weights(difficulty_scores, curriculum_upweight)
# Group by reward quartiles
sorted_indices = np.argsort(rewards)
group_size = len(rewards) // 4
loss = 0
for quartile in range(4):
group_start = quartile * group_size
group_end = (quartile + 1) * group_size
group_indices = sorted_indices[group_start:group_end]
group_rewards = torch.tensor([rewards[i] for i in group_indices])
group_weights = torch.tensor([weights[i] for i in group_indices])
# Higher quartiles are positive examples
if quartile >= 2:
# Positive group: maximize likelihood
group_loss = -torch.log(torch.tensor([model_outputs[i] for i in group_indices]) + 1e-8)
else:
# Negative group: minimize likelihood
group_loss = torch.log(torch.tensor([reference_outputs[i] for i in group_indices]) + 1e-8)
# Weight by curriculum
weighted_loss = (group_loss * group_weights).mean()
loss += weighted_loss
return loss / 4
3. Implement Two-Stage Training
Stage 1: Learn brevity via easy samples. Stage 2: Improve on hard problems.
def two_stage_frugal_training(model, dataset, token_budget=16000):
"""Train with frugal curriculum"""
classifier = ProblemDifficultyClassifier()
# Categorize problems
categorized = classifier.categorize_problems([d['problem'] for d in dataset])
# Stage 1: Learn brevity with heavy easy-sample weighting
print("Stage 1: Learning brevity via easy samples...")
optimizer = torch.optim.Adam(model.parameters(), lr=1e-4)
stage1_data = categorized['easy'] + categorized['medium'] # Mix easy and medium
stage1_weights = compute_curriculum_weights(
[classifier.estimate_difficulty(p) for p in stage1_data],
up_weighting=3.0 # Heavy up-weighting for brevity
)
for epoch in range(3):
for problem, weight in zip(stage1_data, stage1_weights):
# Generate solution with token budget
solution = model.generate(problem, max_tokens=token_budget)
# Reward: correctness + brevity
correct = verify_solution(solution, problem)
length_efficiency = 1.0 / (1.0 + len(solution.split()) / 100)
reward = 0.8 * correct + 0.2 * length_efficiency
# Loss weighted by curriculum
loss = -torch.log(torch.tensor(reward)) * weight
optimizer.zero_grad()
loss.backward()
optimizer.step()
# Stage 2: Improve on hard problems with learned brevity
print("Stage 2: Improving on hard problems...")
stage2_data = categorized['hard']
stage2_weights = compute_curriculum_weights(
[classifier.estimate_difficulty(p) for p in stage2_data],
up_weighting=1.0 # Normal weighting for hard problems
)
for epoch in range(10):
for problem, weight in zip(stage2_data, stage2_weights):
# Generate with token budget (model has learned brevity)
solution = model.generate(problem, max_tokens=token_budget)
# Reward: emphasis on correctness
correct = verify_solution(solution, problem)
length_efficiency = 1.0 / (1.0 + len(solution.split()) / 100)
reward = 0.95 * correct + 0.05 * length_efficiency
loss = -torch.log(torch.tensor(reward)) * weight
optimizer.zero_grad()
loss.backward()
optimizer.step()
return model
4. Evaluate Emergent Brevity
Measure length and accuracy trade-off.
def evaluate_frugal_reasoning(model, test_problems, ground_truth_answers):
"""Evaluate model on length and accuracy"""
results = {
'accuracies': [],
'solution_lengths': [],
'tokens_per_problem': []
}
for problem, gt_answer in zip(test_problems, ground_truth_answers):
# Generate solution
solution = model.generate(problem, max_tokens=16000)
# Check correctness
predicted_answer = extract_answer(solution)
correct = predicted_answer == gt_answer
results['accuracies'].append(correct)
results['solution_lengths'].append(len(solution.split()))
results['tokens_per_problem'].append(len(solution.split()) * 1.3) # Token estimate
# Compute metrics
accuracy = np.mean(results['accuracies'])
avg_length = np.mean(results['solution_lengths'])
efficiency = np.mean(results['tokens_per_problem'])
print(f"Accuracy: {accuracy:.1%}")
print(f"Avg solution length: {avg_length:.0f} words")
print(f"Token efficiency: {efficiency:.0f} tokens/problem")
return results
When to Use:
Hyperparameters:
When NOT to Use:
Pitfalls:
Key Insight: The mechanism works because easy samples teach "fast solution patterns." Once the model learns to solve easy problems quickly, these patterns transfer to hard problems. No explicit length penalty needed.