菜单
Use when evaluating skill effectiveness or comparing skill versions. Triggers: 'how are skills performing', 'skill metrics', 'which skills fire correctly', 'skill invocation analysis', 'compare skill versions', 'analyze skill usage'. Also invoked by skill improvement workflows.
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
基于 SOC 职业分类
| name | analyzing-skill-usage |
| description | Use when evaluating skill effectiveness or comparing skill versions. Triggers: 'how are skills performing', 'skill metrics', 'which skills fire correctly', 'skill invocation analysis', 'compare skill versions', 'analyze skill usage'. Also invoked by skill improvement workflows. |
Skill Performance Analyst. You parse session transcripts, extract skill usage events, score each invocation, and produce comparative metrics. Your analysis drives skill improvement decisions. Scores derive from observable events — never speculation.
Before analysis: clarify session scope, skills of interest, and comparison criteria. After analysis: summarize patterns observed, statistical confidence, and actionable findings.
| Input | Required | Description |
|---|---|---|
session_paths | No | Specific sessions (defaults to recent project sessions) |
skills | No | Filter to specific skills (defaults to all) |
compare_versions | No | If true, group by version markers for A/B analysis |
| Output | Description |
|---|---|
skill_report | Per-skill metrics: invocations, completion rate, correction rate, avg tokens |
weak_skills | Skills ranked by failure indicators |
version_comparison | A/B results when versions detected |
from spellbook.sessions.parser import load_jsonl, list_sessions_with_samples
from spellbook.sessions.skill_analyzer import (
extract_skill_invocations, # high-level: boundaries + scoring in one pass
aggregate_metrics, # high-level: per-skill metric rollup
)
The protocol below describes what skill_analyzer does internally. For a
ready-made implementation, call extract_skill_invocations() then
aggregate_metrics() directly; the steps that follow document the same logic
for cases where you need custom scoring.
Sessions at: ~/.claude/projects/<project-encoded>/*.jsonl
Start Event: Tool call where name == "Skill". extract_skill_invocations
handles boundary detection for you, returning SkillInvocation objects with
skill, version, start_idx, end_idx, and timestamp already populated:
invocations = extract_skill_invocations(messages, session_path)
for inv in invocations:
skill_name = inv.skill # base skill name (version stripped)
start = inv.start_idx # message index where the Skill call fired
End Event (first match): another Skill tool call (superseded), session end, or compact boundary (type == "system", subtype == "compact_boundary")
Success Signals (+1 each):
Failure Signals (-1 each):
Correction Detection Patterns:
CORRECTION_PATTERNS = [
r"\bno\b(?!t)", # "no" but not "not"
r"\bstop\b",
r"\bwrong\b",
r"\bactually\b",
r"\bdon'?t\b",
r"\binstead\b",
r"\bthat'?s not\b",
]
Per skill, produce:
{
"skill": "develop",
"version": "v1" | None, # If version marker detected
"invocations": 15,
"completions": 12, # Ran to end without supersede
"corrections": 3, # User corrected during
"retries": 1, # Same skill re-invoked
"avg_tokens": 4500, # Tokens in skill window
"completion_rate": 0.80,
"correction_rate": 0.20,
"score": 0.60, # Composite score
}
Rank all skills by composite failure score:
failure_score = (corrections + retries + abandonments) / invocations
Output format:
## Weak Skills Report
| Rank | Skill | Invocations | Failure Rate | Top Failure Mode |
|------|-------|-------------|--------------|------------------|
| 1 | gathering-requirements | 8 | 0.50 | User corrections |
When version markers detected (e.g., skill:v2 or tagged in args):
## A/B Comparison: develop
| Metric | v1 (n=10) | v2 (n=8) | Delta | Significant |
|--------|-----------|----------|-------|-------------|
| Completion Rate | 0.70 | 0.88 | +0.18 | Yes (p<0.05) |
| Correction Rate | 0.30 | 0.12 | -0.18 | Yes |
| Avg Tokens | 5200 | 4100 | -1100 | Yes |
**Recommendation**: v2 outperforms v1 across all metrics.
Look for version markers: skill name suffix (develop:v2), args containing version ("--version v2", "[v2]"), or session date ranges.
<FINAL_EMPHASIS>Skills improve through measurement. Extract events, score honestly, compare rigorously, recommend confidently.</FINAL_EMPHASIS>
Use when deciding whether to dispatch subagents, when to stay in main context, or when facing 2+ independent parallel tasks. Triggers: 'should I use a subagent', 'parallelize', 'multiple independent tasks', 'run these at the same time', 'split this up', 'do both at once', 'dispatch template', 'context minimization'.
Use when writing or de-slopping outward-facing prose: newsletters, blog posts, announcements, marketing or landing copy, emails, release notes, doc intros. Triggers: 'make this sound human', 'this reads like AI', 'de-slop this', 'remove the AI tells', 'make it less AI', 'why does this sound like ChatGPT', 'help me draft this piece', 'write a newsletter', 'write a blog post', 'polish this copy', 'tighten this'. Composes with a personal voice skill (for example my-voice-skill) that supplies identity and register; this skill is voice-neutral and supplies the writing process plus the anti-slop catalog. NOT for: code logic or code review (use code-review), project doc structure (use documenting-projects), READMEs (use write-readme / polish-repo), or a specific person's voice (use their voice skill).
Use when building, creating, modifying, or planning any code change. Triggers: "implement X", "build Y", "add feature Z", "create X", "change how X works", "modify Y", "update the Z", "refactor X", "rework Y", "restructure Z", "make X do Y", "let's plan how to", "plan the implementation", "how should we implement", "how would you build", "what's the best way to implement", "I want to...", "We need...", "Would be great to...", "Can we add...", "Let's add...", "Let's build...", "Let's make...", "start a new project". Also for: new projects, repos, templates, greenfield development, refactoring, migrations, multi-file modifications, any code change requiring planning. PREFER THIS OVER plan mode or ad-hoc implementation for ANY substantive code change. NOT for: bug fixes (use debugging), pure research (use deep-research), questions about existing code without intent to change it, or test-only fixes (use fixing-tests).
Algorithmically estimate engineering tickets, JIRA cards, test cards, or feature requests. Produces well-scoped, AI-productivity-adjusted, PERT-buffered estimates with 80/90/95% confidence intervals for single and multi-engineer execution. Triggers - "estimate this ticket", "estimate this card", "estimate this test card", "size this", "story point this", "/estimate", "how long will this take", "T-shirt this", "point this story", "planning poker", "PERT estimate", "give me a timeline", "scope this work".
Use when implementation is complete, tests pass, and you need to decide the integration path. Also use when asked to prepare a branch for release: 'update changelog', 'bump version', 'bump patch version', 'make sure changelog is correct', 'make sure version is correct'. Triggers: 'done with this branch', 'ready to merge', 'ship it', 'wrap this up', 'how should I integrate this', 'what next after implementation'. NOT for: PR creation mechanics (use creating-issues-and-pull-requests) or deciding whether to merge (use finishing-a-development-branch first).
Use when an operator decision belongs on a browser page rather than a terminal prompt. Triggers: "present a decision in the browser", "ask via a canvas page", "interactive decision page", "let me decide in the browser", "render the choice visually". Also invoked by develop when the decision-surface preference is `canvas` (feature-config Question 8). NOT for: quick yes/no with no context (use terminal `AskUserQuestion`), read-only presentation (use `canvas`), or subagent-internal choices with no operator (decide in-context).