Menu
Investigate and triage failed benchmark tasks, analyze root causes, and plan reruns.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Based on SOC occupation classification
Archive old completed benchmark runs to save disk space and speed up scans. Triggers on archive runs, clean up runs, disk space, old runs.
Audit benchmark suites against ABC framework (Task/Outcome/Reporting validity). Checks instruction quality, verifier correctness, reproducibility. Triggers on benchmark audit, audit benchmark, abc audit, task validity.
Verify infrastructure readiness before launching benchmark runs — tokens, Docker, disk, credentials. Triggers on check infra, infrastructure check, ready to run, pre-run check.
Compare benchmark results across agent configurations (baseline, SG_full). Show where configs diverge. Triggers on compare configs, config comparison, which config wins, MCP impact.
Token and cost analysis per run, suite, and config. Shows most expensive tasks and config cost comparison. Triggers on cost report, how much did it cost, token usage, spending.
Generate the aggregate CSB evaluation report from completed Harbor runs. Triggers on generate report, eval report, ccb report, benchmark report.
| name | triage |
| description | Investigate and triage failed benchmark tasks, analyze root causes, and plan reruns. |
Use this skill when the user asks to:
# Triage a specific task result
python3 scripts/analysis/status_fingerprints.py runs/staging/run_dir/task_id/result.json
# List all failures in a run with categorization
python3 scripts/analysis/aggregate_status.py --staging --suite csb_sdlc_debug --failures-only
# Deep analysis of error patterns
grep -r "error\|fail\|timeout" runs/staging/run_dir/*/trajectory.json
# Check task logs and stdout
cat runs/staging/run_dir/task_id/agent/solution.md
/run skill with --rerun flag or custom selection file/run — execute reruns after diagnosis/status — see current failure overview/evaluate — deep analysis of task outputs