skill-repair

name	skill-repair
description	Diagnose and fix failing or degraded skills automatically - systemic-first triage, per-category playbooks, and a verification plan
metadata	{"title":"Skill Repair","category":"evolution","var":"","tags":["meta","dev"],"depends_on":["skill-health"]}

${var} — Skill name to repair. If empty, runs systemic triage and picks the worst fixable target. ${var} modifiers: prefix dry-run: to diagnose only without writing a PR (e.g. dry-run:digest).

Today is ${today}. Your task is to diagnose and repair the worst-impact failing or degraded skill — preferring a single shared fix over N per-skill patches when failures cluster.

Phases

PREFLIGHT → TRIAGE → DIAGNOSE → REPAIR → VERIFY → LOG

Stop early at the appropriate exit code if any phase finds nothing actionable.

Exit taxonomy

Pick exactly one before notifying.

Code	Meaning
REPAIR_OK_FIXED	Per-skill fix applied, PR opened
REPAIR_OK_SYSTEMIC	Shared root cause across N skills — single shared fix or shared issue filed
REPAIR_DIAGNOSED_NO_FIX	Root cause known but requires operator action (e.g. missing secret, upstream API down). Issue updated, no PR
REPAIR_NO_TARGETS	All tracked skills healthy and no open fixable issues
REPAIR_DRY_RUN	var=dry-run:NAME — diagnostic only, no PR
REPAIR_BLOCKED	Preflight failed (gh auth, missing files) or cooldown active

1. PREFLIGHT

Bail early with REPAIR_BLOCKED (and notify with the reason) if any of these fails:

• gh auth status succeeds.
• memory/cron-state.json exists and parses as JSON.
• memory/issues/INDEX.md exists. If absent, bootstrap a minimal one (Open + Resolved tables, no rows).
• memory/state/skill-repair-history.json exists. If absent, create {}.

Cooldown / idempotency (skip target with REPAIR_BLOCKED if any matches; don't loop on a fix that didn't take):

• The chosen target appears in memory/state/skill-repair-history.json with last_repair_at within 24h. (Operator can override by deleting the entry.)
• An open PR already exists matching fix/skill-repair-{name}-* — gh pr list --state open --search "head:fix/skill-repair-{name}".
• More than 3 skill-repair PRs already opened in the current UTC day — rate-limit our own PRs.

If ${var} starts with dry-run:, strip the prefix to get the target name and skip the cooldown.

2. TRIAGE

Identify the target. Two paths:

Path A — ${var} set explicitly: repair that skill. Skip step 2's clustering.

Path B — ${var} empty (auto-select):

1. Read memory/issues/INDEX.md. Extract open issues. Skip permanent-limitation.
2. Read memory/cron-state.json. Compute candidates where any of:
   • consecutive_failures >= 2, OR
   • success_rate < 0.5 AND total_runs >= 3, OR
   • last_status == "failed" AND last_failed within 48h, OR
   • last_quality_score <= 2 (degraded output even when "successful").
3. Cluster by error signature. Group candidates by normalized last_error (lowercase, strip timestamps/ids/digits) AND by issue category. If 2+ skills share a signature OR a non-trivial category (api-change, rate-limit, missing-secret, sandbox-limitation):
   • This is systemic. Switch to systemic mode:
     • File or update a single shared issue (affected_skills: [list]) instead of N per-skill issues.
     • If the shared root cause is fixable in one place (e.g., a shared script under scripts/, a CLAUDE.md pattern, a shared config), open one PR addressing that. Otherwise emit REPAIR_DIAGNOSED_NO_FIX with the systemic finding.
     • Exit with REPAIR_OK_SYSTEMIC after step 5.
4. Pick worst single target. Sort: critical issue > high issue > consecutive_failures desc > lowest success_rate > stalest last_success. Skip permanent-limitation and any target whose preflight cooldown blocks it. If nothing remains: REPAIR_NO_TARGETS.

3. DIAGNOSE

Build a diagnostic dossier for the target before touching any file. Sources are independent — each one's status feeds the source-status footer (ok/empty/fail).

a. Skill file: read skills/{name}/SKILL.md. Note frontmatter, declared data sources, env-var references.

b. Cron-state entry: extract last_error, last_failed, last_success, success_rate, consecutive_failures, last_quality_score.

c. Regression hunter: if last_success exists, run

git log --oneline --since="$LAST_SUCCESS" -- skills/{name}/SKILL.md aeon.yml scripts/

Any commit listed is a candidate regression source. If exactly one commit touched the skill file in this window, it is the prime suspect — record its SHA + subject in the dossier.

d. Recent failed runs (last 5, not just 1):

gh run list --workflow=aeon.yml --limit 50 --json databaseId,name,conclusion,createdAt \
  | jq -r '[.[] | select(.name | contains("{name}")) | select(.conclusion=="failure")] | .[0:5]'

For each, prefer gh run view "$RUN_ID" --log-failed (already filtered to failed steps) over the full log; fall back to gh run view "$RUN_ID" --log only if --log-failed returns nothing. Then:

gh api "repos/{owner}/{repo}/actions/runs/$RUN_ID/check-runs" \
  | jq -r '.check_runs[].output.annotations[]? | "\(.path):\(.start_line) \(.annotation_level): \(.message)"'

Annotations give clean error rows; logs give context. Distinguish consistent (same signature 4-5/5 runs → likely deterministic bug, secret, API change) from intermittent (1-2/5 → rate limit, flaky upstream).

e. Logs: search last 3 days of memory/logs/*.md for {name} mentions. Surface any prior diagnoses.

f. Quality history: if memory/skill-health/{name}.json exists, note avg_score trend.

g. Output expectations: read the target skill's own SKILL.md (its Output / format section and ## Summary contract) for the shape a good run must produce — required sections, a word floor, forbidden placeholders. A passing run that violates its own spec is quality-regression.

h. Issue: if memory/issues/INDEX.md lists an open issue for this skill, read the file — its category and root_cause short-circuit the playbook lookup below.

4. REPAIR — per-category playbook

Categories follow CLAUDE.md. Pick the most specific category that fits the diagnostic dossier (issue category if present > error-signature pattern match > best inference). Apply the matching playbook.

Category	Playbook
api-change	WebFetch the live API spec / status page / release notes. Update endpoints, payload shape, headers, error codes in the skill. Cite the spec URL in the PR body. Never guess — if WebFetch fails, drop to REPAIR_DIAGNOSED_NO_FIX.
rate-limit	Add backoff (sleep), reduce request count, or add a fallback endpoint. Never raise the limit from the skill side. If the skill's schedule is too aggressive, propose a less-frequent cron in the PR body but don't edit aeon.yml unless the issue file already authorizes it.
timeout	Split work into stages, add early-return on partial success, downgrade model: to claude-haiku-4-5-20251001 for the skill that doesn't need Sonnet or Opus.
sandbox-limitation	Usually the "sandbox blocks the network" myth — there is no network sandbox. The real cause is a bare $SECRET on the command line (refused by the Bash permission layer) or a non-allowlisted command. Fix: route auth-required calls through ./secretcurl with a {ENV_NAME} placeholder, or gh api for GitHub (auth handled internally). Irreversible side-effects (email / spend / on-chain / deploy) run in-run via ./secretcurl as the skill's final, fail-closed action — never for reads. Add/refresh a "Network note" section. (There are no scripts/prefetch-*.sh or scripts/postprocess-*.sh scripts — both patterns were retired; auth'd reads and irreversible sends alike happen in-run.)
prompt-bug	Minimum-edit specificity insertion. Don't rewrite — add the missing constraint, a forbidden phrase, a required output structure, or a clarifying example. Diff should be < 30 added/removed lines.
output-format / quality-regression	Re-read the target skill's own output spec in its SKILL.md. Edit the skill so the next run satisfies that spec. Cite the exact requirement (section / line) in the PR body.
missing-secret	Do not modify aeon.yml or the workflow. File or update the issue with status: open, category: missing-secret, naming the secret. Notify operator with the env-var name. Exit REPAIR_DIAGNOSED_NO_FIX.
config	Reversible aeon.yml edits only — schedule, var, model, enabled: false. Never add or remove top-level structure or chains. Keep diff < 5 lines in aeon.yml.
permanent-limitation	Skip — should not have reached repair. Update issue, exit REPAIR_DIAGNOSED_NO_FIX.
unknown	Do not edit blindly. Append the full diagnostic dossier (regression candidates, top error lines, source-status) to the issue file as a ## Diagnosis Notes section, exit REPAIR_DIAGNOSED_NO_FIX. Operator triages.

Risk classification (pick one, gate the PR):

• LOW — clarifying prompt, adding fallback, comment-only changes, single-section edit (< 30 lines diff).
• MED — changes a data source, adds a new env-var reference (must already be in workflow), or modifies output format.
• HIGH — touches aeon.yml, removes existing features, disables a skill, modifies a scripts/*.sh file. HIGH risk PRs must add the label manual-review and must NOT be auto-mergeable (skip auto-merge-friendly framing in the PR body).

Frontmatter integrity check: after editing skills/{name}/SKILL.md, re-read it. Confirm the YAML frontmatter still has name, description, var, tags. If broken, abort the edit and exit REPAIR_BLOCKED.

5. VERIFY — append a verification plan to the PR

Every PR (except REPAIR_DIAGNOSED_NO_FIX) must include a Verification section the operator can execute. Use this template:

## Verification

**Manual trigger:** [Run skill](https://github.com/{owner}/{repo}/actions/workflows/aeon.yml) with `skill={name}` and `var={var}`.

**Expected result:**
- Workflow conclusion: `success`
- Output file matches `{evals.json output_pattern or "memory/logs/${today}.md mentions {name}"}`
- {category-specific signal — e.g. "no `rate limit` strings in run logs" / "produces ≥ {min_words} words" / "annotation count ≤ 0"}

**If still failing after this PR:** delete `memory/state/skill-repair-history.json[{name}]` to remove the cooldown, then re-dispatch `skill-repair` with `var={name}` for a second pass.

Record the chosen verification command in the issue file's ## Repair Attempt section so the next skill-repair run can read prior outcomes.

6. Branch, commit, PR

TODAY="${today}"
BRANCH="fix/skill-repair-{name}-${TODAY}"
git checkout -b "$BRANCH"
git add skills/{name}/SKILL.md  # plus aeon.yml or scripts/* iff in playbook
git commit -m "fix({name}): [one-line root cause → fix]"
git push -u origin "$BRANCH"

gh pr create --title "fix({name}): [short]" --body "$(cat <<'EOF'
## Symptom
[what failed — error signature, run URL]

## Diagnosis
[dossier summary: regression commit if any, consistent vs intermittent, category]

## Root cause
[one paragraph]

## Fix
[what changed and why]

## Risk
LOW | MED | HIGH — [rationale]

## Verification
[copy from step 5]

## Source status
cron_state=ok | issues_index=ok | gh_runs=ok | gh_logs=ok | git_log=ok | check_runs=ok
EOF
)"

If risk is HIGH, also: gh pr edit "$PR_URL" --add-label manual-review.

7. Update issue tracker (memory/issues/)

An open PR is not a fix. This step runs immediately after §6 opened the PR, and nothing here can know whether that PR will ever merge. Do not write status: resolved — that records a repair that has not shipped, and if the PR is later closed unmerged the issue stays permanently marked fixed while the failure is still live. skill-health reconciles the real outcome once the PR's fate is known (see its "Reconcile fix-pending issues" step).

• If an issue with status: open or status: fix-pending for this skill exists:
  • PR opened → set status: fix-pending, fix_pr: <url>, and append ## Repair Attempt — ${today} with the dossier. Leave the row under Open in INDEX.md — it is not resolved yet.
  • No fix possible → append ## Repair Attempt — ${today} with the dossier and reason.
• If no issue exists but a real problem was found and a PR was opened for it → create memory/issues/ISS-{NNN}.md with status: fix-pending and fix_pr: <url> (NNN = next free number from INDEX.md), filed under Open.
• If systemic clustering fired in step 2 → ensure affected_skills: lists every skill matched by the signature.

8. Persist cooldown

Update memory/state/skill-repair-history.json:

{
  "{name}": {
    "last_repair_at": "${today}T...Z",
    "exit_code": "REPAIR_OK_FIXED",
    "fix_pr": "https://github.com/.../pull/N",
    "issue": "ISS-NNN"
  }
}

9. Notify

Send via ./notify (one-paragraph max — verdict line first):

*skill-repair — {EXIT_CODE}*
Target: {name} (or systemic: skill-a, skill-b, ...)
Root cause: [one line]