eval-evolve-operator

name	eval-evolve-operator
description	Govern eval and evolve work inside the prune repo. Use when a no-context agent needs to understand eval/evolve entrypoints, inspect local report artifacts, judge whether a manual eval or evolve run is warranted, review uncommitted working-tree changes as possible evolve output, and promote durable workflow conclusions into tracked docs, rules, and skills without committing raw artifacts.

Eval / Evolve Operator Skill

Use this when the job is to supervise the repo's local eval and evolve loops rather than answer one narrow code question.

Start from repo state

• Read docs/README.md first.
• Then read eval/README.md, docs/evolve.md, and skills/prune/SKILL.md.
• Run python3 skills/eval-evolve-operator/scripts/inspect_state.py --repo-root <repo> when you need a compact view of current git dirtiness plus the newest eval and evolve artifacts.
• Use --json as a quick alias for machine-readable output, or --format json when you want the mode spelled out explicitly.
• When the next implementation step is already clear, use python3 skills/eval-evolve-operator/scripts/write_handoff.py ... to write a schema-valid handoff artifact instead of drafting ad hoc JSON by hand. Use --from-file <existing-handoff.json> when you want to reuse a prior handoff as a base and only override a few fields, --dry-run to preview the generated handoff without writing the file, or --validate-only to check an existing handoff file against the schema.
• Add --harness <claude|codex|surfwind> when the uncertainty is provider-specific and the latest overall artifact would be misleading. Surfwind requires stricter JSON output guidance (see decision-rules.md).
• Treat uncommitted working-tree changes as first-class evidence. They may be the most important trace of a prior evolve pass even when nothing was committed.

Operating stance

• Keep eval/reports/, eval/repos/checkouts/, and evolve/runs/ as local artifacts. Read them, cite them, and compare them, but do not commit them.
• Do not auto-commit code or docs just because an evolve run modified them. Let git status surface those changes as reviewable working-tree evidence.
• Stay in the review, routing, and decision layer by default. Use manual eval or evolve runs only when they answer a real uncertainty about product fit, harness quality, or next-step selection.
• Do not treat this operator as the default place to implement prune, eval, or evolve changes. When implementation is needed, route it to evolve/run_once.py or to another implementation agent, then review the result here.
• When routing work into evolve/run_once.py, prefer an explicit JSON handoff that matches evolve/handoff-schema.json. Use skills/eval-evolve-operator/scripts/write_handoff.py to create it, then invoke the runner with --handoff-file <path> so the execution side receives a real scope contract instead of only a prose reminder.
• If the same session must both review and implement, make the handoff explicit: finish the operator diagnosis first, write down the narrow implementation target and verification, switch modes deliberately, and then come back here for verification and promotion.
• When you need repository inspection, use prune by default and follow skills/prune/SKILL.md.

Read eval signals

When reviewing process-summary.json from eval or evolve runs:

• prune_command_count vs raw_inspection_command_count shows whether agents followed the skill
• For eval command-mix risk (raw_inspection_command_count, failed_command_count), read the latest solve run first; use latest eval only when no solve run exists
• Zero raw inspection with non-zero prune counts indicates good skill adherence
• High prune counts with zero raw inspection may indicate over-cautious inspection (check if agent stopped early)
• failed_command_count points to broken commands or provider-side issues
• Compare these signals across runs to judge whether prompt or harness changes help

Choose the next action

• If the working tree is dirty, inspect those paths before planning another evolve run. Dirty paths may already contain the next useful step.
• When eval signals show zero raw inspection but high prune counts, check whether the agent stopped too early due to over-cautious following of the skill.
• If the latest change touches src/, eval/, evolve/, skills/prune/, or output and prompt contracts, consider whether a targeted eval is now missing.
• If the latest eval solve exists but no newer evaluate pass scores it, decide whether you need that verdict before iterating further.
• If the latest evolve run blocked, open the highest-signal artifact first instead of retrying blindly.
• If the latest evolve run carried a handoff, inspect handoff_scope_audit_status and any out-of-scope paths before trusting that run as a clean implementation pass.
• If a failure looks harness- or provider-boundary-specific, inspect the boundary contract and the provider-side source or docs before tweaking prompts or retrying broad runs.
• When local code changes already look correct but behavior still depends on the harness/provider interaction, prefer one targeted real run over a broad batch.
• If the next improvement is already obvious and local, prefer emitting a concrete handoff (goal, why_now, start_points, allowed_scope, constraints, verification) instead of turning this operator into the implementation agent.

Load these references on demand

• Read references/entrypoints.md for exact runners, artifact locations, and review order.
• Read references/decision-rules.md for trigger heuristics, promotion boundaries, and how to interpret dirty git state.
• Read references/layer-contracts.md when you need the repo-local contract for product, eval, operator, evolve, and the operator-to-evolve handoff.