| name | sunset-review |
| description | Use this skill when the user wants to identify unused, near-zero-use, or stale skills/agents/commands in the plugin surface so they can be demoted or retired. Combines agent-dispatch telemetry (start-events only) with static reference scanning, classifies every surface item into Active / Investigate / Demote / Retire, and emits a Markdown report plus JSON sidecar. NEVER auto-deletes — surfaces candidates for human decision. Quarterly cadence. <example>Context: The plugin surface has grown and the maintainer wants to prune dead weight. user: "/sunset-review" assistant: "Running the sunset walk — classifying skills, agents, and commands by usage telemetry + static refs, grouped by Retire / Demote / Investigate / Active. No item is deleted automatically; I'll surface Retire/Demote candidates for your decision." <commentary>The user wants a usage-driven prune candidate list; this skill runs the read-only walker, presents grouped verdicts, and writes a sidecar — it never deletes.</commentary></example> |
Sunset Review Skill
Identify which skills, agents, and commands in the plugin surface are still earning their keep, and which are candidates to Demote (downgrade docs/tier) or Retire (remove). The skill is advisory: it produces a ranked candidate list and a sidecar artifact. It never deletes anything.
Why this is a distinct skill (not /repo-audit)
/repo-audit answers "does this repo match the ecosystem baseline?" — a pass/fail compliance check. /sunset-review answers a different question — "which parts of OUR surface are unused?" — with different inputs (dispatch telemetry + static ref scanning), a different cadence (quarterly, not per-session), and a different output (prune candidates, not compliance status). Folding it into repo-audit would muddy both.
Core data contract (read this before trusting any verdict)
The walker — scripts/lib/sunset/walker.mjs — is read-only and was built against the following grep-verified telemetry facts. Do not override them:
- Both agent dispatch and skill-invocation telemetry are now consumed.
subagents.jsonl records agent dispatches; skill-invocations.jsonl records Skill tool selection events (L1 telemetry, epic #645). Agents are assessed by agent-dispatch counts; skills are assessed by real selection counts from skill-invocations.jsonl, supplemented by static reference scanning. Commands still have no invocation telemetry and are assessed by static reference scanning only. Skills with zero invocations in the window receive a zero-count (not null), so the low-coverage guard still applies and can downgrade Retire → Investigate when the telemetry window is short.
- Only
event === "start" records count. agent_type is null on every stop event. A stop event must never mark an agent cold. The walker filters to start events.
- Telemetry only spans ~18 days. A 90-day window cannot be satisfied by 18 days of data. MANDATORY GUARDRAIL: when
coverageDays < windowDays, every Retire verdict is downgraded to Investigate and meta.lowConfidence is set true. Retiring on sub-window data is unsafe — treat any cold finding as "investigate", not "delete".
- Zero ≠ near-zero. A never-dispatched agent (e.g.
memory-proposal-collector, a by-design reference doc) is a Retire candidate; a once-dispatched agent (db-specialist, ui-developer at n=1) is a Demote candidate. The walker distinguishes them.
Verdict tiers
| Verdict | Meaning | Action posture |
|---|
| Active | Agent dispatch above floor, OR skill invoked-by-command / command invokes-a-live-skill, OR ≥2 non-boilerplate refs | Keep |
| Investigate | Coverage < window (low-confidence), OR conflicting signals — default-safe bucket | Manual review |
| Demote | Single dispatch (n=1), OR a skill with a single cross-ref and no command linkage | Consider downgrading docs/tier |
| Retire | dispatch===0 AND nonBoilerplateRefs===0 AND coverage>=window | Strong remove candidate (verify first) |
Phases
Phase 1 — Resolve config + window
- Read Session Config for any
sunset-review.window-days override; default to 90 (DEFAULT_WINDOW_DAYS).
- The window is the period over which dispatch counts are tallied. Keep the default unless the operator has a reason to narrow it.
Phase 2 — Run the walker (JSON mode)
Run the read-only walker and capture its JSON:
node scripts/lib/sunset/walker.mjs --json --window-days 90 > /tmp/sunset-walk.json
Exit 0 = walk completed (cold findings are exit 0, not an error). Exit 1 = bad args / surface dir missing. Exit 2 = system error. The walker writes no files — it is pure analysis.
Optionally scope to one kind for focused review:
node scripts/lib/sunset/walker.mjs --json --kind agent
Phase 3 — Classify & present (NEVER auto-delete)
- Group the
items[] array by verdict, ordered Retire → Demote → Investigate → Active.
- Lead with
meta.lowConfidence. If true, state plainly: "telemetry covers only N days (< window); all Retire candidates were downgraded to Investigate — do not retire anything this run."
- For each Retire/Demote candidate, show its
reasons[] and signals{} so the operator can sanity-check the verdict against their own knowledge (a low-traffic-but-load-bearing item should be spared).
Phase 4 — Emit report + sidecar
Mirror the repo-audit sidecar convention. Write both:
- Markdown report:
.orchestrator/metrics/sunset-review-<unix-timestamp>.md — the grouped, human-readable verdict list with reasons.
- JSON sidecar: the walker's full JSON output, persisted alongside for trend tracking.
mkdir -p .orchestrator/metrics
The sidecar is the durable record; the Markdown is the readable summary.
Skill Health (advisory) — render a per-skill health advisory block into the report.
- Source: per-skill verdicts come from
scoreSkillHealth() in scripts/lib/skill-health/score.mjs, computed over the L2 join (scripts/lib/skill-health/join.mjs) plus any optional L3 judgments — the same telemetry the walker already surfaces on item.signals.judge. The mechanical/CI surface for this data is harness-audit category9 ("Skill-Health Surfacing").
- Rendering: add a
## Skill Health section to the Markdown report. List each skill that has sufficient samples with its verdict (insufficient signal | trigger description unclear | instructions wrong) and its diagnosis string. Skills below the sample threshold (MIN_SAMPLES_FOR_VERDICT, ~20) are NOT scored per-skill — collapse them into a single line: "Insufficient signal (N skills)".
- Firewall (MANDATORY): the health advisory NEVER edits any skill file and NEVER pushes a skill toward Retire or Demote on health data alone. It may only annotate the existing verdict, or trigger the existing Active→Investigate downgrade the walker already applies — never an escalation. This mirrors the walker's advisory-only judge firewall: annotate or downgrade-to-Investigate, never escalate toward Retire/Demote.
- Default-empty: when telemetry is absent or insufficient (the common case today), render exactly "Skill Health: insufficient signal across all skills (no action)". This is a healthy state, not a finding — do not surface it as a candidate.
Phase 5 — Cadence note (the SKILL writes last-run, NOT the walker)
- Sunset review runs quarterly. The walker is stateless and writes no runtime file — recording the last-run timestamp is this skill's responsibility, so a session-start nudge can fire when a quarter has elapsed.
- After writing the report, record the run time (e.g. into
.orchestrator/metrics/sunset-review-last-run.json with an ISO timestamp). This keeps the walker free of mutable runtime state and concurrency surface.
Draft-issue creation (coordinator-side only — AUQ-004)
If the operator wants to file removal/demotion issues for the surfaced candidates, that decision must be made by the coordinator via AskUserQuestion — a dispatched agent cannot call AskUserQuestion (AUQ-004). When this skill runs inside a dispatched agent, it surfaces the candidate list and defers issue creation to the coordinator thread. Never auto-file issues and never auto-delete surface items.
Output Summary
- Grouped verdict list printed to the user (Retire/Demote/Investigate/Active).
- Markdown report at
.orchestrator/metrics/sunset-review-<timestamp>.md.
- JSON sidecar (walker output) alongside.
- Last-run timestamp recorded for the quarterly cadence nudge.
- Zero deletions. Zero auto-filed issues.
See Also
scripts/lib/sunset/walker.mjs — the read-only walker (CLI + exported functions).skills/repo-audit/SKILL.md — the sibling compliance audit (different question)..claude/rules/cli-design.md — JSON-first / exit-code contract the walker follows..claude/rules/ask-via-tool.md — AUQ-004 (agents cannot call AskUserQuestion).
