harness-benchmark-builder

name	harness-benchmark-builder
description	Build and maintain the benchmark task set for the harness evolution plugin.

harness-benchmark-builder Skill

Purpose

Curate the benchmark suite used to gate candidate promotion. This skill keeps task coverage balanced across audit, governance, workflow, and held-out regression checks.

Steps

1. Inspect scripts/harness_core/benchmarks/tasks/.
2. Confirm the suite still covers:
   • harness audit tasks
   • runtime governance tasks
   • instruction hygiene tasks
   • candidate PR workflow tasks
   • held-out regression tasks
3. Add or refine YAML tasks without reducing held-out coverage below 4 tasks.
4. Dry-run the suite:

   python ${CLAUDE_PLUGIN_ROOT}/scripts/harness_core/benchmark_runner.py \
     --harness-dir $CLAUDE_PROJECT_DIR \
     --tasks-dir ${CLAUDE_PLUGIN_ROOT}/scripts/harness_core/benchmarks/tasks \
     --run-type baseline \
     --dry-run
   

Constraints

• Held-out tasks are regression sentinels; do not repurpose them for new behavior.
• Favor deterministic assertions over narrative evaluation.
• Keep task YAML readable and scoped to a single intent.