菜单
Correct reasoning errors with minimal data collection by using an oracle to surgically fix only erroneous steps in existing trajectories. Use binary classification loss on rectified pairs with implicit KL regularization to prevent knowledge forgetting.
默认使用会先检查来源的 Prompt；你也可以切换为直接命令，或下载本地副本。
决定是否安装前，请先阅读 SKILL.md，以及 SkillsMP 当前展示的配套文件。
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
直接命令不会经过审查 Prompt；运行前请先检查来源。
npx skills add https://github.com/ADu2021/skillXiv --skill surgical-post-training-error-correction命令会保持在同一行。复制前请横向滚动并检查完整内容。
想先保存到本地？可下载 SkillsMP 当前能够提供的文件。
Convert arXiv papers into ready-to-use agent skills using category-aware extraction. First classifies the paper into one or more of 11 research categories, then applies a specialized extraction pipeline for each category — because different types of papers produce different types of usable knowledge. A single paper can yield multiple skills if it spans categories. Use this skill whenever the user wants to turn a paper into a skill, extract practical techniques from research, build a skill library from papers, convert arXiv papers into reusable agent instructions, or batch-process multiple papers into skills. Also trigger when someone asks about extracting actionable knowledge from papers, making research practical for LLM agents, or systematically converting academic contributions into structured agent capabilities.
Establish regret bounds for behavior cloning with discretized actions combining statistical error and quantization error terms. Prove smoothness requirements for safe quantizer design, show that learning-based quantizers fail these requirements, and propose model-based augmentation to reduce error dependence from H² to H.
Dynamically allocate LoRA ranks per-layer during fine-tuning instead of using fixed uniform ranks. Learn optimal rank for each layer and subject via variational framework with discretized exponential distribution, reducing memory footprint while maintaining fidelity and text-alignment.
基于 SOC 职业分类
正在显示 SKILL.md
| name | surgical-post-training-error-correction |
| title | Surgical Post-Training: Cutting Errors, Keeping Knowledge |
| version | 0.0.2 |
| engine | skillxiv-v0.0.2-claude-opus-4.6 |
| license | MIT |
| url | https://arxiv.org/abs/2603.01683 |
| keywords | ["Post-Training","Error Correction","Data Rectification","Binary Classification","Reasoning Alignment"] |
| description | Correct reasoning errors with minimal data collection by using an oracle to surgically fix only erroneous steps in existing trajectories. Use binary classification loss on rectified pairs with implicit KL regularization to prevent knowledge forgetting. |
Standard post-training data collection requires extensive high-quality annotation. Surgical Post-Training (SPoT) takes a different approach: starting with existing model-generated trajectories, use a stronger oracle model to surgically correct only the erroneous reasoning steps while preserving the original trajectory structure and style. This minimal-edit approach dramatically reduces annotation burden while maintaining knowledge from pretraining.
The core insight is that error correction should be surgical, not wholesale. Most of a trajectory is correct; fixing only the broken parts minimizes distribution shift. Combined with binary classification objectives and implicit KL regularization through reward-based loss, SPoT achieves consistent improvements with 4,000-10,000 rectified pairs instead of the 100,000+ examples typical approaches require.
SPoT operates through three coordinated mechanisms:
Step 1: Generate candidate erroneous and correct pairs
Sample trajectories from the model, identify failures, and use oracle for correction.
def generate_rectified_pairs(base_model, oracle_model, prompt_batch,
num_samples_per_prompt=2):
"""Generate erroneous model outputs and oracle corrections."""
rectified_pairs = []
prompt prompt_batch:
candidates = [base_model.generate(prompt) _ (num_samples_per_prompt)]
candidate candidates:
is_correct, error_steps = oracle_model.evaluate(candidate)
is_correct (error_steps) > :
corrected = oracle_model.fix_steps(candidate, error_steps)
rectified_pairs.append({
: candidate,
: corrected,
: error_steps
})
rectified_pairs
Step 2: Filter pairs by trajectory similarity
Ensure rectified pairs differ only minimally to avoid distribution shift. Use Longest Common Subsequence (LCS) to measure overlap.
def lcs_ratio(s1, s2):
"""Compute longest common subsequence length / max(len(s1), len(s2))."""
m, n = len(s1), len(s2)
dp = [[0] * (n + 1) for _ in range(m + 1)]
for i in range(1, m + 1):
for j in range(1, n + 1):
if s1[i - 1] == s2[j - 1]:
dp[i][j] = dp[i - 1][j - 1] + 1
else:
dp[i][j] = max(dp[i - 1][j], dp[i][j - 1])
lcs_length = dp[m][n]
return lcs_length / max(m, n)
# Filter pairs: keep only those with >60% token-level overlap
min_similarity_ratio = 0.60
filtered_pairs = []
for pair in rectified_pairs:
tokens_incorrect = pair['incorrect'].split()
tokens_correct = pair['correct'].split()
similarity = lcs_ratio(tokens_incorrect, tokens_correct)
if similarity >= min_similarity_ratio:
filtered_pairs.append(pair)
# else: too much divergence; skip to avoid distribution shift
print(f"Kept {len(filtered_pairs)} pairs after similarity filtering")
Step 3: Tokenize and prepare for training
Convert trajectories to token sequences for model training.
def prepare_training_batch(filtered_pairs, tokenizer, max_length=512):
"""Tokenize rectified pairs for training."""
incorrect_ids = []
correct_ids = []
attention_masks = []
for pair in filtered_pairs:
# Tokenize both incorrect and correct versions
incorrect_tokens = tokenizer.encode(
pair['incorrect'],
max_length=max_length,
truncation=True,
padding='max_length',
return_tensors='pt'
)
correct_tokens = tokenizer.encode(
pair['correct'],
max_length=max_length,
truncation=True,
padding='max_length',
return_tensors='pt'
)
incorrect_ids.append(incorrect_tokens['input_ids'])
correct_ids.append(correct_tokens['input_ids'])
attention_masks.append(correct_tokens['attention_mask'])
return {
'incorrect_ids': torch.cat(incorrect_ids),
'correct_ids': torch.cat(correct_ids),
'attention_mask': torch.cat(attention_masks)
}
Step 4: Design binary classification loss with KL regularization
The core loss treats correct vs. incorrect as classification, regularized by KL to reference model.
def compute_spt_loss(model, reference_model, batch, temperature=1.0):
"""
SPoT loss = Binary Classification + KL Regularization
Classification: maximize likelihood of correct responses, minimize incorrect
KL: prevent divergence from reference model via elastic tether
"""
incorrect_ids = batch['incorrect_ids']
correct_ids = batch['correct_ids']
attention_mask = batch['attention_mask']
# Forward pass: get log probabilities for both sequences
with torch.no_grad():
ref_logits_correct = reference_model(correct_ids)['logits']
ref_logits_incorrect = reference_model(incorrect_ids)['logits']
logits_correct = model(correct_ids, attention_mask=attention_mask)['logits']
logits_incorrect = model(incorrect_ids, attention_mask=attention_mask)['logits']
# Binary classification objective:
# - Maximize likelihood of correct trajectory
# - Minimize likelihood of incorrect trajectory
loss_correct = -torch.nn.functional.log_softmax(logits_correct, dim=-1).mean()
loss_incorrect = torch.nn.functional.log_softmax(logits_incorrect, dim=-1).mean()
classification_loss = loss_correct - loss_incorrect # Rank correct above incorrect
# KL regularization: constrain divergence from reference model
# This acts as an "elastic tether" preventing catastrophic forgetting
kl_loss = torch.nn.functional.kl_div(
torch.nn.functional.log_softmax(logits_correct / temperature, dim=-1),
torch.nn.functional.softmax(ref_logits_correct / temperature, dim=-1),
reduction='batchmean'
)
# Combined loss with KL weight
beta = 0.1 # KL regularization strength
total_loss = classification_loss + beta * kl_loss
return total_loss
Step 5: Training loop with convergence monitoring
Train the model with rectified pairs, monitoring both accuracy improvement and knowledge retention.
def train_spt(model, reference_model, train_dataloader,
epochs=3, learning_rate=2e-5):
"""Fine-tune model with Surgical Post-Training."""
optimizer = torch.optim.AdamW(model.parameters(), lr=learning_rate)
for epoch in range(epochs):
total_loss = 0.0
num_batches = 0
for batch in train_dataloader:
optimizer.zero_grad()
# Compute loss
loss = compute_spt_loss(model, reference_model, batch)
# Backward and optimize
loss.backward()
torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)
optimizer.step()
total_loss += loss.item()
num_batches += 1
avg_loss = total_loss / num_batches
print(f"Epoch {epoch + 1}: Loss = {avg_loss:.4f}")
# Early stopping: if loss plateaus, reduce learning rate
if epoch > 0 and abs(avg_loss - prev_loss) < 1e-4:
learning_rate *= 0.5
for param_group in optimizer.param_groups:
param_group['lr'] = learning_rate
prev_loss = avg_loss
return model
Step 6: Evaluation and knowledge retention check
Verify that fine-tuned model improves on error correction while retaining general knowledge.
def evaluate_spt(model, test_prompts, oracle_model, baseline_model):
"""Evaluate SPoT: accuracy improvement and knowledge retention."""
correct_before = 0
correct_after = 0
total = len(test_prompts)
for prompt in test_prompts:
# Baseline model output
response_baseline = baseline_model.generate(prompt)
is_correct_before, _ = oracle_model.evaluate(response_baseline)
# Fine-tuned model output
response_ft = model.generate(prompt)
is_correct_after, _ = oracle_model.evaluate(response_ft)
if is_correct_before:
correct_before += 1
if is_correct_after:
correct_after += 1
improvement = (correct_after - correct_before) / total * 100
accuracy_after = correct_after / total * 100
print(f"Accuracy improvement: {improvement:+.1f}%")
print(f"Final accuracy: {accuracy_after:.1f}%")
return improvement, accuracy_after
Hyperparameter Selection:
When to Use:
When NOT to Use:
Common Pitfalls: