rstack-skill-evaluator

name	rstack-skill-evaluator
description	Benchmark and iteratively improve agent skills with Codex CLI or Claude Code by generating eval cases, comparing skill-guided vs baseline runs, grading outcomes, and recording actionable reports under skills-test/{skill-name}.
metadata	{"dependencies":["skill-creator"],"internal":true}

Rstack Skill Evaluator

A repo-specific compatibility layer on top of skill-creator. Reuse its Test / Improve / Benchmark concepts, JSON schemas, grading guidance, and eval viewer, but select the executor for the current environment.

Select the executor

• When the user requests Codex or codex is the available CLI, read references/codex-cli.md and follow it. Its instructions override Claude-specific commands and subagent mechanics in skill-creator.
• Otherwise, follow skill-creator directly.

Do not invoke skill-creator/scripts/run_eval.py, run_loop.py, or improve_description.py in Codex mode. Those scripts shell out to claude -p and test Claude-specific skill discovery. Provider-neutral utilities such as quick_validate.py, aggregate_benchmark.py, and eval-viewer/generate_review.py can be reused after validating the installed dependency version.

Targeting a skill

If the user hasn't named a target, ask. Skills live under skills/ (production) and .agents/skills/ (internal-only).