| name | harness-benchmark-builder |
| description | Build and maintain the benchmark task set for the harness evolution plugin. |
harness-benchmark-builder Skill
Purpose
Curate the benchmark suite used to gate candidate promotion. This skill keeps
task coverage balanced across audit, governance, workflow, and held-out
regression checks.
Steps
- Inspect
scripts/harness_core/benchmarks/tasks/.
- Confirm the suite still covers:
- harness audit tasks
- runtime governance tasks
- instruction hygiene tasks
- candidate PR workflow tasks
- held-out regression tasks
- Add or refine YAML tasks without reducing held-out coverage below 4 tasks.
- Dry-run the suite:
python ${CLAUDE_PLUGIN_ROOT}/scripts/harness_core/benchmark_runner.py \
--harness-dir $CLAUDE_PROJECT_DIR \
--tasks-dir ${CLAUDE_PLUGIN_ROOT}/scripts/harness_core/benchmarks/tasks \
--run-type baseline \
--dry-run
Constraints
- Held-out tasks are regression sentinels; do not repurpose them for new behavior.
- Favor deterministic assertions over narrative evaluation.
- Keep task YAML readable and scoped to a single intent.