skill-optimizer

name	skill-optimizer
description	Guides validation-gated improvement of an existing SecondBrain skill using scored rollout evidence, deterministic evidence-to-edit proposals, bounded add/delete/replace edits, materialized candidate artifacts, rejected-edit memory, durable run state, Antahkarana-backed gate outcomes, and protected slow-update guidance. Use when the user wants to optimize a skill, evolve a SKILL.md, improve skill performance, or turn replay failures into a better reusable skill artifact.
version	1.4.0
triggers	["optimize a skill","improve skill performance","evolve a SKILL.md","tune this skill","turn replay failures into skill edits","record skill optimization in Antahkarana","propose skill edits from evidence","build candidate skill artifact"]
negative_triggers	["create a brand new skill from scratch","uninstall a skill","optimize model weights"]
allowed_tools	[]
required_tools	[]
trust_level	local
safety_class	standard
examples	["Draft a validation-gated plan for improving an existing skill from replay evidence."]
tests	["sb skills validate --path skills --strict","pytest -q tests/skills/test_skill_optimizer_plan.py tests/skills/test_skill_optimizer_run.py tests/skills/test_skill_evidence.py tests/skills/test_skill_patching.py tests/skills/test_skill_selection_gate.py tests/skills/test_skill_antahkarana_bridge.py tests/skills/test_skills_cli_lifecycle.py"]
performance	{"notes":"Best when the skill has scored rollouts, replay cases, or benchmark evidence."}

Skill Optimizer

Purpose

Improve an existing skill as a trainable text artifact while keeping the target agent, harness, and evaluator fixed.

Instructions

1. Identify the target skill, its current SKILL.md, and the task domain it should improve.
2. Collect scored evidence before editing: replay cases, rollout traces, benchmark cases, verifier output, or a human-reviewed score table.
3. Split evidence into train, selection, and test roles. Use train evidence to propose edits, selection evidence to accept or reject candidates, and test evidence only for final reporting.
4. Normalize scored evidence from JSON, JSONL, or YAML before proposing edits. Every record should have a task/case id, score or verdict, and verifier feedback or an explicit failure pattern.
5. Produce small patch-style edits only: append, insert after exact text, replace exact text, or delete exact text.
6. Separate failure reflection from success reflection. Failure edits fix recurring gaps; success edits preserve working behavior not already encoded.
7. Rank edits by systematic impact, complementarity, generality, and actionability. Apply only the current edit budget.
8. Materialize candidate SKILL.md artifacts separately from the live skill. Do not overwrite the live skill before held-out selection accepts the candidate.
9. Accept a candidate skill only if it strictly improves the held-out selection score. Reject ties.
10. Record accepted and rejected gate outcomes into the Antahkarana stack when a stack database is available: accepted candidates leave successful Karma and procedural Chitta evidence; rejected candidates leave a regret, lesson, next action, and procedural memory.
11. Record rejected edits with the score delta and observed failure pattern so later edits avoid repeating harmful directions.
12. Keep long-horizon guidance in a protected slow-update section:

<!-- SLOW_UPDATE_START -->
Durable epoch-level guidance goes here.
<!-- SLOW_UPDATE_END -->

Step-level edits must not modify that protected section.

Planning Command

Use the deterministic planning surface before proposing edits:

sb skills optimize-plan <skill-name> --path skills --evidence <rollouts.jsonl> --json

The plan encodes the default optimization loop, validation gate, edit budget, protected slow-update markers, evidence requirements, and recommended local checks for the selected skill.

Start a durable optimization run before applying or gating candidates:

sb skills optimize-start <skill-name> \
  --path skills \
  --evidence <rollouts.jsonl> \
  --run-dir out/skill-optimizer/<skill-name>-run \
  --json

This writes run.json, plan.json, a baseline SKILL.md snapshot, candidate and rejected-edit directories, and a best-artifact pointer. Check progress with:

sb skills optimize-status out/skill-optimizer/<skill-name>-run --json

Apply structured edits through the dry-run-first patching surface:

sb skills propose-edits <skill-name> \
  --path skills \
  --evidence <train-rollouts.jsonl> \
  --run-state out/skill-optimizer/<skill-name>-run \
  --json

propose-edits normalizes scored evidence, ranks repeated failure patterns ahead of success patterns, writes a bounded patch proposal, and records the proposal summary in run.json when --run-state is supplied.

Review the proposal, then dry-run it:

sb skills apply-edits <skill-name> --path skills --edits proposal-001.yaml --json

Only add --write after reviewing the apply report and candidate validation. Patch files must contain an edits list or patch.edits list with operations append, insert_after, replace, or delete.

After the dry-run is clean, materialize a candidate artifact for held-out selection:

sb skills build-candidate <skill-name> \
  --path skills \
  --edits out/skill-optimizer/<skill-name>-run/proposals/proposal-001.yaml \
  --run-state out/skill-optimizer/<skill-name>-run \
  --json

This writes candidates/candidate-001.SKILL.md, an apply report, and a compact candidate summary in run.json.

Gate candidate scores before promotion:

sb skills gate-candidate <skill-name> \
  --path skills \
  --baseline-score 0.72 \
  --candidate-score 0.75 \
  --candidate out/skill-optimizer/<skill-name>-run/candidates/candidate-001.SKILL.md \
  --run-state out/skill-optimizer/<skill-name>-run \
  --json

Ties and regressions are rejected. When a candidate is rejected, write a rejected-edit buffer so future edit attempts can avoid repeating the same direction:

sb skills gate-candidate <skill-name> \
  --path skills \
  --baseline-score 0.72 \
  --candidate-score 0.70 \
  --edit-report edit-apply-report.json \
  --rejected-buffer rejected-edit-buffer.json \
  --json

When optimizing inside a SecondBrain workspace with an Antahkarana database, also record the gate result into the stack:

sb skills gate-candidate <skill-name> \
  --path skills \
  --baseline-score 0.72 \
  --candidate-score 0.70 \
  --rejected-buffer rejected-edit-buffer.json \
  --antahkarana-db .secondbrain/state/runtime.db \
  --run-state out/skill-optimizer/<skill-name>-run \
  --json

Use the returned antahkarana.trace_id to inspect the persisted Karma, Chitta, and Sankalpa trace. Rejections should create a Karma regret and next action; acceptances should preserve a procedural memory of what improved. Use the returned run summary to confirm counters and the current best artifact.

Acceptance Checklist

• The target model and harness stay fixed during comparison.
• The skill has strict-valid frontmatter, clear triggers, examples, tests, and performance metadata.
• Candidate edits are grounded in repeated evidence, not a single anecdote.
• Evidence proposal files are generated from train evidence, not the held-out selection/test split.
• Candidate artifacts are materialized and validated before held-out gating.
• The selection gate uses a held-out score and rejects ties.
• Antahkarana recording is used when the optimizer is running inside a stack-backed workspace.
• run.json records every gate and points to the current best artifact.
• Rejected edits and failure patterns are preserved as negative feedback.
• The final handoff names the best skill artifact and the exact verification commands.