菜单
Enables continuous self-improvement through corpus-grounded self-play. Challenger mines difficult examples from document corpus for Reasoner to solve. External corpus prevents task stagnation that plagues closed-loop self-play. Achieves 8.9% math, 9.8% general reasoning improvements.
默认使用会先检查来源的 Prompt；你也可以切换为直接命令，或下载本地副本。
决定是否安装前，请先阅读 SKILL.md，以及 SkillsMP 当前展示的配套文件。
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
直接命令不会经过审查 Prompt；运行前请先检查来源。
npx skills add https://github.com/ADu2021/skillXiv --skill spice-corpus-self-play命令会保持在同一行。复制前请横向滚动并检查完整内容。
想先保存到本地？可下载 SkillsMP 当前能够提供的文件。
正在显示 SKILL.md
基于 SOC 职业分类
| name | spice-corpus-self-play |
| title | SPICE: Self-Play In Corpus Environments Improves Reasoning |
| version | 0.0.2 |
| engine | skillxiv-v0.0.2-claude-opus-4.6 |
| license | MIT |
| url | https://arxiv.org/abs/2510.24684 |
| keywords | ["Self-play","Reasoning","Corpus Grounding","RL","Curriculum Learning"] |
| description | Enables continuous self-improvement through corpus-grounded self-play. Challenger mines difficult examples from document corpus for Reasoner to solve. External corpus prevents task stagnation that plagues closed-loop self-play. Achieves 8.9% math, 9.8% general reasoning improvements. |
Self-play RL struggles with improvement plateaus because agents exhaust limited task spaces. SPICE grounds self-play in a real-world document corpus, enabling continuous challenge generation at the frontier of capability.
The corpus provides the rich, near-inexhaustible signal necessary for sustained reasoning improvement.
Two complementary roles:
The Challenger creates automatic curriculum using corpus, addressing the fundamental limitation of closed-loop self-play: limited task diversity.
Implement Challenger that mines tasks from corpus:
class ChallengerModel:
def __init__(self, llm, corpus):
self.llm = llm
self.corpus = corpus
def mine_task(self, reasoner_capability_level):
"""Generate task at boundary of reasoner capability."""
# Sample documents from corpus
candidates = self.corpus.sample(num_candidates=100)
difficulty_scores = []
for doc in candidates:
# Extract problem from document
problem = self.llm.extract_problem(doc)
# Estimate difficulty relative to reasoner capability
difficulty = self.estimate_difficulty(problem, reasoner_capability_level)
difficulty_scores.append((problem, difficulty))
# Select problems at frontier of capability
target_difficulty = reasoner_capability_level +
frontier_problems = [
p p, d difficulty_scores
(d - target_difficulty) <
]
frontier_problems[] frontier_problems candidates[]
():
problem_emb = .llm.embed(problem)
similar_solved = []
solved_problem .reasoner_history:
similarity = cosine_similarity(
problem_emb,
.llm.embed(solved_problem)
)
similar_solved.append(similarity)
avg_similarity = (similar_solved) / (similar_solved) similar_solved
- avg_similarity
Implement Reasoner training with corpus-generated tasks:
class ReasonerModel:
def __init__(self, llm):
self.llm = llm
self.solved_problems = []
def solve(self, task):
"""Attempt to solve mined task."""
solution = self.llm.generate(task, max_tokens=500)
return solution
def get_reward(self, task, solution):
"""Verify solution quality."""
# Use external verifier if available
# E.g., Python execution, ground truth, heuristics
return verify_solution(task, solution)
def train_step(self, task, solution, reward):
"""RL update from corpus-generated task."""
# Policy gradient update
log_prob = self.llm.compute_log_prob(task, solution)
loss = -log_prob * reward
# Backward and update
self.optimizer.zero_grad()
loss.backward()
self.optimizer.step()
# Track solved problems for difficulty estimation
if reward > threshold:
self.solved_problems.append(task)
Implement the co-training loop:
def train_spice(challenger, reasoner, corpus, num_rounds=100):
"""Self-play loop with corpus grounding."""
for round_idx in range(num_rounds):
# Challenger mines task at frontier
task = challenger.mine_task(reasoner.capability_level)
# Reasoner attempts task
solution = reasoner.solve(task)
# Evaluate solution
reward = reasoner.get_reward(task, solution)
# Train both models
reasoner.train_step(task, solution, reward)
# Update Challenger's understanding of reasoner capability
challenger.update_reasoner_profile(task, reward)
if round_idx % 100 == 0:
# Evaluate on held-out benchmarks
benchmark_score = evaluate_on_benchmark(reasoner)
print(f"Round {round_idx}: Benchmark = {benchmark_score:.2%}")
# Corpus prevents task exhaustion
# Challenger can always mine new problems
| Parameter | Recommendation |
|---|---|
| Corpus size | 100K+ documents |
| Difficulty tolerance | ±0.2 relative to capability |
| Solved problem memory | Track last 1000 problems |
| Reward threshold | Task-specific |
When to use:
When NOT to use:
Common pitfalls:
Reference: SPICE on arXiv