菜单
Run a single benchmark task locally to verify a fix. Uses haiku for speed. Triggers on quick rerun, rerun task, verify fix, test task.
默认使用会先检查来源的 Prompt；你也可以切换为直接命令，或下载本地副本。
决定是否安装前，请先阅读 SKILL.md，以及 SkillsMP 当前展示的配套文件。
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
直接命令不会经过审查 Prompt；运行前请先检查来源。
npx skills add https://github.com/sourcegraph/CodeScaleBench --skill quick-rerun命令会保持在同一行。复制前请横向滚动并检查完整内容。
想先保存到本地？可下载 SkillsMP 当前能够提供的文件。
基于 SOC 职业分类
正在显示 SKILL.md
| name | quick-rerun |
| description | Run a single benchmark task locally to verify a fix. Uses haiku for speed. Triggers on quick rerun, rerun task, verify fix, test task. |
| user-invocable | true |
Run a single benchmark task with minimal settings to verify a fix works.
User provides:
benchmarks/csb_sdlc_pytorch/sgt-005sgt-005If user gave a task name, find the benchmark path:
cd ~/CodeScaleBench
# Find the task definition
find benchmarks -type d -name "TASKNAME" | head -5
Ask the user or infer from context:
none — baseline, no MCP toolsbase — Sourcegraph base tools (keyword search)deepsearch — Sourcegraph full (keyword + deep search)Default to none (baseline) for fastest verification.
cd ~/CodeScaleBench
# Load credentials
source .env.local 2>/dev/null ||
PYTHONPATH=
configs/_common.sh
ensure_fresh_token
cd ~/CodeScaleBench
# Baseline (no MCP) — fastest
BASELINE_MCP_TYPE=none harbor run \
--path benchmarks/<suite>/<task_name> \
--agent-import-path agents.claude_baseline_agent:BaselineClaudeCodeAgent \
--model anthropic/claude-haiku-4-5-20251001 \
--jobs-dir /tmp/quick-rerun \
-n 1 \
--timeout-multiplier 1.0 \
2>&1 | tee /tmp/quick-rerun.log
For MCP-Full:
BASELINE_MCP_TYPE=sourcegraph_full harbor run \
--path benchmarks/<suite>/<task_name> \
--agent-import-path agents.claude_baseline_agent:BaselineClaudeCodeAgent \
--model anthropic/claude-haiku-4-5-20251001 \
--jobs-dir /tmp/quick-rerun \
-n 1 \
2>&1 | tee /tmp/quick-rerun.log
# Find the most recent task output
LATEST=$(ls -td /tmp/quick-rerun/*/ 2>/dev/null | head -1)
if [ -z "$LATEST" ]; then
# Check for batch dir layout
LATEST=$(ls -td /tmp/quick-rerun/*/*/ 2>/dev/null | head -1)
fi
echo "Task dir: $LATEST"
# Check result.json
if [ -f "$LATEST/result.json" ]; then
python3 -c "
import json
data = json.load(open('$LATEST/result.json'))
exc = data.get('exception_info')
vr = data.get('verifier_result') or {}
rewards = vr.get('rewards') or {}
reward = rewards.get('reward', rewards.get('score'))
if exc:
exc_msg = exc.get('exception_message', exc.get('message', str(exc)[:100])) if isinstance(exc, dict) else str(exc)[:100]
print(f'ERRORED: {exc_msg}')
elif reward is not None and reward > 0:
print(f'PASS (reward={reward})')
else:
print(f'FAIL (reward={reward})')
"
else
echo "No result.json found — task may still be running or crashed early"
echo "Log tail:"
tail -20 /tmp/quick-rerun.log 2>/dev/null
fi
If the user wants a more realistic test:
# Use opus for production-grade verification
--model anthropic/claude-opus-4-5-20251101
If the fix was to the verifier/test.sh and you just want to re-check scoring:
cd <task_dir>
bash benchmarks/<suite>/<task_name>/tests/test.sh
rm -rf /tmp/quick-rerun