skill-eval

name	skill-eval
description	Evaluate all workspace-hub skills for structural validity, content quality, cross-reference integrity, and registry consistency. Runs 18 checks across critical, warning, and info severity levels with actionable fix suggestions.
type	reference
version	1.0.0
category	development
last_updated	"2026-01-29T00:00:00.000Z"
capabilities	["structural_validation","content_quality_analysis","cross_reference_integrity","report_generation"]
tools	["Bash","Read","Glob","Grep"]
related_skills	["skill-creator","verification-loop"]
requires	[]
see_also	[]
tags	[]

Skill Eval

Quick Start

# Full evaluation of all skills
uv run .Codex/skills/development/skill-eval/scripts/eval-skills.py

# JSON output
uv run .Codex/skills/development/skill-eval/scripts/eval-skills.py --format json

# Single skill
uv run .Codex/skills/development/skill-eval/scripts/eval-skills.py --skill testing-tdd-london

# Only critical issues
uv run .Codex/skills/development/skill-eval/scripts/eval-skills.py --severity critical

When to Use

• Auditing skill quality after bulk creation or migration
• Pre-release validation of the skills library
• CI/CD quality gate for skill changes
• Identifying broken cross-references after renaming skills
• Checking compliance with v2 SKILL.md template format

Core Concepts

Evaluation Dimensions

The evaluator runs 18 checks organized into three severity levels:

Critical (blocks skill usage):

• YAML frontmatter exists and parses
• Required fields present: name, description

Warning (degrades quality):

• Version follows semver, category matches directory
• Required content sections present (Quick Start, When to Use, etc.)
• Code blocks in key sections, no TODO/FIXME markers
• Cross-references in related_skills resolve to real skills

Info (improvement opportunities):

• Uses v2 template format, has optional sections (Metrics, etc.)
• No duplicate skill names across the library

Report Output

Reports include:

• Summary with pass/fail counts and percentages
• Issues grouped by severity with counts
• Per-category breakdown
• Top 10 most common issues
• Per-skill details with actionable fix suggestions

Usage Examples

Full Evaluation

uv run .Codex/skills/development/skill-eval/scripts/eval-skills.py

Output:

================================================================
  SKILL EVALUATION REPORT
  Generated: 2026-01-29T14:30:00+00:00
================================================================

SUMMARY
----------------------------------------
  Total skills evaluated:  230
  Passed (no critical):    187  (81.3%)
  Warnings only:           102  (44.3%)
  Critical failures:        43  (18.7%)