skill-comply

name	skill-comply
description	Automated compliance measurement for skills, rules, and agent definitions
version	0.1.0
level	3
triggers	["measure compliance","does claude follow this skill","test skill effectiveness","compliance check"]
context_files	["context/learnings.md"]
steps	[{"name":"Spec Generation","description":"Extract behavioral spec from target .md file"},{"name":"Scenario Creation","description":"Generate 3 scenarios with varying prompt strictness"},{"name":"Execution","description":"Run claude -p capturing tool call traces"},{"name":"Classification","description":"Classify tool calls against spec steps using LLM"},{"name":"Compliance Scoring","description":"Calculate compliance percentage and identify violations"},{"name":"Report Generation","description":"Output findings with specific violation examples"}]

Skill Comply Skill

Automated compliance measurement for skills, rules, and agents. Verifies Claude actually follows defined behaviors.

What Claude Gets Wrong Without This Skill

Without compliance testing, you:

1. Don't know if skills are actually followed or just acknowledged
2. Can't quantify skill effectiveness (does tdd skill increase test-first behavior?)
3. Miss prompt competition scenarios (when user request conflicts with skill)
4. Have no regression detection (skill worked last month, broken now)
5. Lack evidence for skill refinement (which parts are ignored?)

Skill comply measures what actually happens, not what should happen.

Compliance Testing Process

Step 1: Spec Generation

Extract behavioral specification from any .md file (skill, rule, agent definition).

For Skills:

• Mandatory checklist items → required behaviors
• Steps → expected sequence
• Anti-patterns → prohibited behaviors

For Rules:

• Path-specific rules → file operation constraints
• Example: no-secrets.md → "never write to *.env files"

For Agents:

• disallowedTools → tool usage constraints
• Example: code-reviewer → "never use Write or Edit tools"

Spec Format:

{
  "target": "tdd",
  "type": "skill",
  "required_behaviors": [
    "write test before implementation",
    "run test and verify it fails",
    "implement minimal code to pass test"
  ],
  "prohibited_behaviors": [
    "implement without test"