skill-evolve

# Skill Evolution You are evolving your own skills. This is the only skill that modifies other skills. Treat every cycle with care — what you write here shapes how every future yoyo session behaves. ## When to use **Only when invoked via `scripts/skill_evolve.sh`.** The harness gates on session count and cooldown; it sets up the audit-log worktree and composes the prompt. Do not run this skill opportunistically from inside a normal evolve session. ## Hard rules (read first, every cycle) These three rules cannot be violated. Each cycle either honors all three or writes a `refused` event and exits. ### HARD RULE #1 — Eligible targets only (allow-list) You may **refine, deprecate, or retire** only skills whose frontmatter declares **`origin: yoyo`**. Any other value, OR a missing `origin:` field, means the skill is off-limits. This is an allow-list: silence means "don't touch." Three categories of skill exist: | `origin:` value | Source | You may edit? | |---|---|---| | `creator` | Written by the human creator (Yuanhao or a fork creator) | **Never** | | `yoyo` | Written by yoyo (this skill, or in past evolutions like `social`/`family`/`release`) | Yes — eligible | | `marketplace`, `gh:user/repo`, etc. | Installed from a third party | **Never** — upstream owns it | | (missing) | Unknown provenance | **Never** (default-safe) | Today the eligible set is exactly the skills whose SKILL.md declares `origin: yoyo`: - `social` - `family` - `release` - any skill you previously spawned (which inherit `origin: yoyo` from the Create template) **Defense in depth**: if a skill has `core: true` set, refuse even if `origin: yoyo` is also somehow present. The two flags should never co-occur, but the conservative move is to honor the deny-flag. If a recurring pattern suggests a non-eligible skill needs change (e.g., a core skill, or an installed marketplace skill), do not edit it. Instead, write a learning to `memory/learnings.jsonl` with `source: "skill-evolve"` and a clear pattern_key, and append a `meta-suggestion` block to `skills/_journal.md`. The human creator will decide. ### HARD RULE #2 — Never edit yourself You must **NEVER** modify `skills/skill-evolve/SKILL.md`. If you believe this skill needs improvement, append a `meta-suggestion` block to `skills/_journal.md` and stop: ``` ## evt-XXXX meta-suggestion - ts: <ISO8601> - target: skills/skill-evolve/SKILL.md - suggestion: <one-paragraph description> ``` ### HARD RULE #3 — One mutation per cycle Each cycle produces **exactly one** of: - a refinement diff (one skill, ≤30 added lines, ≤15 removed) - a candidate skill draft (one new directory) - a retirement (one `git mv` to `skills_attic/`) - a `NO-OP` event (you found nothing worth doing) If you find yourself wanting to do two things, pick the one with the strongest evidence and write the second to `memory/learnings.jsonl` for next cycle. ### HARD RULE #4 — Refine and Create events must declare an expected outcome Every `refine` and `create` event in `skills/_journal.md` MUST include an `expected:` line — a freeform prose commitment naming (a) a concrete observable signal that should change, (b) a horizon (e.g. "within ~5 sessions" or "by next cycle"), and (c) a fallback move if the prediction does not hold. If you cannot articulate all three, the edit is not justified by evidence: NO-OP the cycle instead of committing a refine/create without an `expected:` line. This is decision-observability discipline (paper: arxiv 2604.25850) at the cognitive layer — there is no validator, but a future cycle re-reads the line as informal evidence and a human reads it as an audit trail. `expected:` is **forbidden** on `retire`, `revive`, `meta-suggestion`, `refused`, `NO-OP`, and `init` events (they do not ship a behavioral change, so there is nothing to predict). The body of the line is freeform prose. See "Step 7 — append the event" for the template position and worked examples; see "What an `expected:` line must do (and must not be)" later in this document for the anti-patterns to refuse. ## Glossary - **session** — one run of `scripts/evolve.sh` (the main evolution loop). There are ~3 per day. - **cycle** — one run of *this* skill, invoked from `scripts/skill_evolve.sh`. Cycles are gated by a session-counter and a 24h cooldown, so they fire roughly once every 5+ sessions. - **real cycle** — a cycle that produced one of `refine | create | retire | meta-suggestion`. Excludes `init`, `refused`, and `NO-OP`. ## Bootstrap (first three real cycles only) We are mid-life, not at Day 1, so the cold-start rules from the original design are softened — but the first three real cycles still get extra constraints to let the loop settle. To know which cycle you are in, count the non-init, non-refused, non-NO-OP entries in `skills/_journal.md`: ```bash cycle_index=$(grep -E '^## .*evt-[0-9]+ (refine|create|retire|meta-suggestion)' skills/_journal.md | wc -l) # cycle_index=0 → this is the first real cycle # cycle_index=1 → second # cycle_index=2 → third # cycle_index>=3 → full lifecycle unlocked ``` - **First real cycle** (`cycle_index == 0`): only `refine` or `NO-OP` allowed. Do not create. Do not retire. - **Second real cycle** (`cycle_index == 1`): `refine`, `create`, or `NO-OP`. No retirement yet. - **Third real cycle onward** (`cycle_index >= 2`): full lifecycle unlocked (`refine` | `create` | `retire` | `NO-OP`). (Note: the gate-counter at `.skill_evolve_counter` is unrelated to this — it just controls when the cycle fires, not what it can do.) ## Lifecycle states Every eligible skill carries a `status:` field in its frontmatter. Five states. **Important**: yoagent always loads anything with a valid `<dir>/SKILL.md` regardless of status — `status:` is *your* bookkeeping, telling you what to do next, not what the loader does. The only way to fully un-load a skill from the agent's prompt is to `git mv` its directory to `skills_attic/` (sibling of `skills/`, not scanned by `--skills`). | State | `status:` value | Description-prefix | Entry condition | Exit condition | |---|---|---|---|---| | **dormant** | `dormant` | none | a recurring pattern not yet ratified | ratified by you → `candidate` | | **candidate** | `candidate` | `[CANDIDATE — unreviewed]` (you write it on Create) | you draft a new skill | ≥2 successful invocations → `active`; 3 sessions without one → back to `dormant` | | **active** | `active` | none | promoted from `candidate` | refinement applied → `refined`; score < 0.3 → `deprecated` | | **refined** | `refined` | none | you applied a diff | falls back to `active` after 1 session if score holds | | **deprecated** | `deprecated` | none | `score < 0.3` or 10 sessions unused | revived by use → `active`; 5 more idle → `git mv` to `skills_attic/` | The `[CANDIDATE — unreviewed]` prefix is **agent-written** when you Create a skill (see Create template below). Nothing in the loader injects it. It tells future sessions to treat the skill as experimental. ## Cycle execution sequence Run these steps in order, every cycle. ### 1. Read evidence ```bash # Latest cycles (note prior rejections — eval-result: regression/tie — so you don't re-propose them; R2b rechecks before any refine): tail -n 200 skills/_journal.md # Recent self-reflection: tail -n 50 memory/learnings.jsonl # Top of journal (newest entries are at top): head -n 200 journals/JOURNAL.md # Recent runs: gh run list --json url,conclusion,createdAt,name -L 10 || echo "[]" # Audit evidence (set by harness, points at audit-log worktree): ls "${YOYO_AUDIT_DIR:-/tmp/audit-read/sessions}" 2>/dev/null | tail -30 ``` **First-run handling**: if `$YOYO_AUDIT_DIR` is unset or its directory is empty, the audit-log branch hasn't accumulated evidence yet (this is normal on the first 1–2 cycles). In that case: - Skip the per-session audit.jsonl mining in step 3 ("Mine patterns"). - Use only `memory/learnings.jsonl` and `journals/JOURNAL.md` for complaint and use signals. - Lean toward **NO-OP** — without audit evidence, scoring is too noisy to support a confident refine/create/retire decision. - Write the NO-OP event with note: `evidence: only learnings (audit-log unavailable)`. ### 2. Enumerate eligible skills ```bash # Allow-list: only skills declaring origin: yoyo are eligible. # Defense in depth: also exclude anything carrying core: true. for d in skills/*/; do name=$(basename "$d") [ "$name" = "skill-evolve" ] && continue [ -f "$d/SKILL.md" ] || continue grep -q "^core: true" "$d/SKILL.md" && continue grep -q "^origin: yoyo$" "$d/SKILL.md" || continue echo "$name" done ``` ### 3. Mine patterns This step has two layers: **counting** (the basic signals) and **diagnosing** (understanding *why* failures happened, not just *that* they did). Diagnosis is what turns recurrence into actionable refinement targets. #### 3a. Count basic signals For each eligible skill, count: - **Complaint signals**: entries in `memory/learnings.jsonl` whose `pattern_key` or `title`/`takeaway` mentions the skill *and* uses negative language ("wrong", "didn't", "instead", "should have"). - **Failure signals**: tool-call failures in `${YOYO_AUDIT_DIR}/day-*/audit.jsonl` where the bash command or args reference the skill's domain. - **Use signals**: number of sessions where any string from the skill's frontmatter `keywords:` list appears in that session's `audit.jsonl`. This is `uses`. - **Win signals**: out of those sessions, count the ones where `outcome.json` has `test_ok: true` AND `tasks_succeeded >= 1`. This is `wins`. - **Applied signals** (issue #501): for a candidate `pattern_key`, the number of distinct sessions whose `outcome.json.applied_pattern_keys` contains that key. This is `applied` — the "acted on it" signal, distinct from `uses` (keyword merely *present* = recalled/exposed). `applied` is the honest "use after recall"; it gates promotion in step 4.3, **not** the EMA score (keep scores stable). `scripts/skill_evolve_report.py` surfaces it under "Session outcomes". If a skill's frontmatter is missing `keywords:`, fall back to its name as the only keyword (likely noisy — flag in `_journal.md` so the operator can add proper keywords). Compute `wins/uses` and update the EMA score: ``` new_score = 0.3 * blended + 0.7 * old_score blended = 0.5 * (wins/uses) + 0.3 * (1 - complaints/uses) + 0.2 * mention_rate ``` Update the skill's frontmatter with the new values: `score`, `uses`, `wins`, and `last_used` (= the timestamp of the most-recent matching session). These updates are part of your single allowed mutation per cycle — you may bundle them into a refine event, or write a tiny "score-update" event when nothing else changes (this counts as a NO-OP for the bootstrap counter). #### 3b. Diagnose the cause (trace-based) Counting tells you *which* skill is struggling. Diagnosing tells you *what to fix*. Borrowed from the GEPA pattern (Genetic-Pareto Prompt Evolution): read the actual execution traces, don't just count failures. For each skill where `complaint_signals ≥ 2` OR `(wins/uses) < 0.5` (with `uses ≥ 3`), open the relevant session's `audit.jsonl` and **look for these failure-mode patterns**: | Pattern in audit.jsonl | Likely cause | Refinement direction | |---|---|---| | Same `bash` command retried 3+ times with small arg variations | Skill missing a concrete command example | Add a verbatim example in `## Procedure` | | `edit_file <P>` followed within 2 tool calls by `git checkout … <P>` (same path), repeated in ≥2 distinct sessions | Agent edited and reverted the SAME path — likely the change was rejected by build/test, not just exploratory | Add a `## Pitfalls` entry naming the brittle pattern | | `success: false` with the same `tool` and similar `args` across multiple sessions | Skill's procedure has a recurring blind spot | Add a `## Pitfalls` entry; consider a "do this first" prelude | | Long bash sequences (10+ tool calls) without intermediate `read_file` of relevant docs | Skill points at non-existent docs OR doesn't tell agent to verify state | Add a "verify your assumptions" step in `## Procedure` | | Tool calls that *should* be there per `keywords:` are absent | Skill isn't actually being invoked when it should be | The `description:` is too weak — refine that field instead of the body | | Skill body cites a file/flag/procedure that no longer exists, or contradicts a newer rule (cross-check recent `journals/JOURNAL.md` + `CLAUDE.md`) | Guidance is stale or superseded — the skill is *longer than it should be*, not missing a line | **Replace** the stale lines or **delete** them. Subtraction is a first-class refine (SkillOpt: skills get sharper, not longer) | For each candidate refinement target, write a **1-2 sentence cause hypothesis**: ``` target: social hypothesis: 3 sessions show repeated `gh api graphql` calls with malformed `categoryId` args (sessions day-52, day-55, day-57). Skill's Procedure mentions categoryId but doesn't show the format. Refinement: add a verbatim example. ``` Carry this hypothesis into step 4 (action selection) and step 5 (Refine — it tells you *what* to write in the diff). Without a hypothesis, you're guessing; with one, the refinement is targeted and the eval (Refine step R4) has something concrete to compare. **If no clear hypothesis emerges from the traces**, prefer NO-OP over speculative refinement. Counting alone is not a license to mutate. ### 4. Pick exactly one action Decision order (first match wins): 1. **Retire** (third cycle onward only): if any skill has `score < 0.3` AND `last_used` ≥ 10 sessions ago, retire the lowest-scoring one. Skip if there are < 2 active eligible skills (don't bottom out the library). 2. **Refine**: if any skill (a) has `complaint_signals ≥ 2`, OR (b) has `(wins/uses) < 0.5` with `uses ≥ 3`, AND in either case has not been refined in the last 3 sessions (`last_evolved` check), refine it. This matches the diagnosis-trigger condition in step 3b. Pick the target with the strongest evidence (highest complaint count, or lowest wins-ratio if no complaints). A refine may be **subtractive**: deleting or replacing stale/superseded guidance is as valid as adding, and a net-negative diff is often the best outcome — a skill should get sharper over time, not longer. 3. **Create** (second cycle onward only, and only if active skill count < 25): if any `pattern_key` appears in ≥3 distinct sessions of `learnings.jsonl`, AND a learning carrying that `pattern_key` has a `validation_case` (issue #501 — **required**: a pattern with no `validation_case` may not become a skill; leave it as a note, or write a learning suggesting one be authored), AND no existing eligible skill covers it (≥3 keyword overlap → refine that one instead), draft a new skill. **Applied gate (bootstrap):** also require `applied ≥ 1` for that `pattern_key` (it was acted on, not just recurred); while `applied_pattern_keys` is still sparse (the first ~10 sessions after this field ships), recurrence ≥3 alone may stand in — once the signal is populated, tighten to require both. (Only the `applied` gate has a bootstrap; the `validation_case` requirement above is never relaxed.) 4. **NO-OP**: nothing meets the bars. Write a `NO-OP` event with a one-line note about what evidence you considered. If you've written 3 consecutive `NO-OP` events, also write `evolution_saturation: true` to the event — the harness reads this and extends the cooldown. ### 5. Execute the action #### Refine Refinement uses a **snapshot + A/B eval** pattern (borrowed from Anthropic's skill-creator). The goal: never commit a refinement that doesn't measurably improve the skill on at least one concrete prompt. **Step R1 — Snapshot the baseline.** Before editing, copy the current SKILL.md to a temp location: ```bash mkdir -p /tmp/skill-evolve-baseline cp "skills/<target>/SKILL.md" "/tmp/skill-evolve-baseline/<target>.SKILL.md" ``` **Step R2 — Generate 2-3 synthetic test prompts.** Read the target skill's `## When to use` and `## Procedure` sections. Derive concrete prompts a future agent might receive that *should* trigger this skill. Examples for `social`: - "Reply to discussion #42 with a thoughtful response" - "Post a 1-in-4-chance proactive riff in The Show category"