メニュー
Benchmark agent skills by generating eval cases, comparing skill-guided vs baseline runs, and recording artifacts under skills-test/{skill-name}.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
| name | rstack-skill-evaluator |
| description | Benchmark agent skills by generating eval cases, comparing skill-guided vs baseline runs, and recording artifacts under skills-test/{skill-name}. |
| metadata | {"dependencies":["skill-creator"],"internal":true} |
A thin repo-specific layer on top of skill-creator. For workflow (Test / Improve / Benchmark modes), JSON schemas, grading, and viewer details, defer to skill-creator's own SKILL.md and references/schemas.md.
If the user hasn't named a target, ask. Skills live under skills/ (production) and dev-skills/ (internal-only).
For skill <name>, two paths are tracked in git; everything else under skills-test/ is gitignored:
+--------------------------------------+----------------------------------+
| Tracked path | Purpose |
+--------------------------------------+----------------------------------+
| skills-test/<name>/evals/evals.json | eval definitions |
| skills-test/<name>/report.md | human-readable run summary |
+--------------------------------------+----------------------------------+
Workspaces, raw run outputs, and fixtures may live anywhere — under skills-test/<name>/ or an OS scratch dir — as long as report.md references the path so a reader can find them.
report.mdCommitted Markdown summary of the latest run. At minimum cover: setup (model, skill version / commit ref, date), aggregate pass rate / tokens / wall time for with_skill vs without_skill, per-eval breakdown, and pointers to the raw artifacts.
Debug Rstack ecosystem CI failures and attribute the real source PR or downstream change. Always use this skill when the user mentions Rspack eco-ci, rstack-ecosystem-ci, a suite turning red, a downstream regression, a green-to-red pivot, canary bisect, @rspack-canary/core, or daily eco-ci triage — even if they only ask "why is this suite failing", "which PR broke it", or "is this Rspack's fault". Use it to avoid over-blaming the first Rspack commit that appears red in status data.
Write or revise release blog posts for product releases. Use when the user asks to draft, polish, restructure, or adapt a release announcement.
Create or update draft GitHub release notes, or output organized Markdown when draft creation is unavailable. Use for release notes, draft releases, release PR checks, npm staged publishing checks, and optional highlights.
Generate or maintain Rspress documentation for a project. Use whenever the user wants to create a new Rspress v2 docs site, add docs for user-facing feature work or PRs, maintain a dedicated Rspress docs project in a monorepo, or prevent stale Rspress v1 scaffolds/version drift before documentation work. For full v1-to-v2 migration, hand off to the rspress-v2-upgrade skill.
Audit and modernize RstackJS/Rspack ecosystem repositories to the current shared infrastructure baseline: Rslib ESM or dual builds, Rslint recommended rules, Rstest test tooling, Node 20+ support where appropriate, TypeScript 6 and tsgo where compatible, concise README/AGENTS docs, release workflow cleanup, unused dependency removal, and infra PR commit conventions. Use when updating rstackjs repositories, copying infra patterns from maintained exemplars, or reviewing package/tooling consistency.
Use when asked to create a pull request for this repository. It helps the PR follow the repository's branch safety rules, title convention, pull request template, and concise English writing style.