gspdev-audit

name	gspdev-audit
description	Verify GSP pipeline integrity — agent/skill contracts, installer correctness, runtime compatibility, version sync, and template coherence. Internal development tool for GSP maintainers.
allowed-tools	["Read","Glob","Grep","Bash","WebFetch","WebSearch","Agent"]
argument-hint	[focus] e.g. 'all', 'contracts', 'installer', 'runtime', 'versions', 'templates'
disable-model-invocation	true

GSP internal integrity checker for maintainers. Verifies that the framework's moving parts stay consistent as the codebase evolves. This is NOT the user-facing `/gsp-doctor` (which checks `.design/` project health) — this checks the GSP *source code* itself.

Source layout:

• gsp/skills/ — 34 skills (SKILL.md files, gsp-* prefixed dirs, with domains/ and references/ siblings)
• gsp/agents/ — 11 agents (gsp-*.md stub files, methodology in skills)
• gsp/templates/ — config, state, brief, roadmap templates
• bin/install.js — multi-runtime installer
• VERSION, package.json — version sources

Runtime compatibility reference: ${CLAUDE_SKILL_DIR}/../gspdev-runtime-compat/references/baseline.md

Run a comprehensive integrity audit and report issues. Every check produces PASS, WARN, or FAIL with actionable fix suggestions.

Step 0: Run automated tests

Run the test suite first — it covers versions, contracts, installer, runtime, and templates:

bash dev/scripts/audit-tests.sh $ARGUMENTS

The script accepts: all (default), versions, contracts, installer, runtime, templates.

Review the output. If all tests pass, report the clean result. If any tests fail or warn, investigate each issue using the deeper analysis steps below.

Step 1: Parse scope

$ARGUMENTS determines which checks to run:

• all or empty — run everything
• contracts — agent↔skill contract checks only
• installer — installer correctness checks only
• runtime — runtime compatibility checks (uses gspdev-runtime-compat baseline)
• versions — version sync checks only
• templates — template coherence checks only

Step 2: Version Sync (V)

Two version sources must agree:

cat VERSION
node -e "console.log(require('./package.json').version)"

V1: Version agreement — both match → PASS, mismatch → FAIL.

V2: CHANGELOG coverage — CHANGELOG.md has an entry for the current version → PASS, missing → WARN.

V5: Template config versions — gsp/templates/branding/config.json and gsp/templates/projects/config.json version fields must match VERSION → PASS, any mismatch → FAIL.

V6: CLAUDE.md counts match filesystem — skill and agent counts referenced in CLAUDE.md (both the source layout table and the installer table) must match actual filesystem counts → PASS, any stale count → FAIL.

Step 3: Contract Checks (C)

Verify that skills and agents reference each other correctly.

C3: Every skill that spawns agents references valid agents

For each gsp/skills/*/SKILL.md (skip get-shit-pretty), extract agent references (patterns: gsp-{name}). Check each referenced agent exists in gsp/agents/.

• All references valid → PASS
• Missing agents → FAIL

C4: Agent tool lists are valid

Read each agent in gsp/agents/. Extract tools: frontmatter. Verify each tool name is a valid Claude Code tool: Read, Write, Edit, Bash, Glob, Grep, WebFetch, WebSearch, Agent, NotebookEdit, TodoWrite, AskUserQuestion, Skill.

• All valid → PASS
• Unknown tools → WARN

C5: No orphan agents

Every agent in gsp/agents/ is spawned by at least one skill.

• All referenced → PASS
• Orphan agents → WARN

C6: Agent frontmatter has required fields

Each agent must have name:, description:, and tools: in frontmatter.

• All present → PASS
• Missing fields → FAIL

C7: Skill frontmatter has required fields

Each skill must have name: and description: in frontmatter.

• All present → PASS
• Missing fields → FAIL

C9: User-invocable skills have user-invocable: true

Every skill in gsp/skills/ (except get-shit-pretty, which is the entry point with user-invocable: false) must have user-invocable: true in frontmatter. Without it, Claude Code won't list the skill in the slash-command menu.

• All present → PASS
• Missing → FAIL with list of skills

C10: Update skill aligned with installer

Verify gsp-update/SKILL.md references all installer runtime flags (--claude, --opencode, --gemini, --codex, --local, --global, --all), current bundle layout (templates/), and doesn't reference legacy bundle paths in "what gets replaced" sections. Catches drift when installer evolves but update skill doesn't.

C8: Claude-only field usage matches known set

Canary test — grep agents for memory:, background:, hooks:, isolation:, skills:, mcpServers:. Compare against expected list (gsp-project-builder.md, gsp-project-reviewer.md). WARN if set changes so developer verifies converters handle new fields.

Step 4: Installer Checks (I)

I1: Installer syntax validity

node -c bin/install.js

PASS if exit 0, FAIL if syntax error.

I2: Source skill count

Verify ≥20 skills exist in gsp/skills/.

I3: Source agent count

Verify ≥10 agents exist in gsp/agents/.

I5: Bundle directories present

Verify gsp/templates exists.

I6: package.json files field

Everything in the files list should exist on disk.

I7: Codex skills → .agents/skills/

Verify getCodexSkillsDir function returns .agents path.

I8: Tool mapping objects present

Verify claudeToOpencodeTools, claudeToGeminiTools, claudeToCodexTools objects exist.

I9: All conversion functions present

Verify all 5 converter functions exist in the installer.

I10: All body replacement functions present

Verify applyOpencodeBodyReplacements, applyGeminiBodyReplacements, applyCodexBodyReplacements exist.

I11: Agent converters strip Claude-only single-line fields

Grep both convertClaudeToOpencodeAgent and convertClaudeToGeminiAgent for startsWith('memory:'), startsWith('background:'), startsWith('isolation:') with continue. FAIL if any stripping pattern is missing.

I12: Agent converters strip Claude-only multi-line blocks

Grep both agent converters for hooks:, skills:, mcpServers: handling and verify inSkipBlock state machine exists. FAIL if block-stripping logic is missing.

I13: Body replacements handle Skill tool rename

Grep each apply*BodyReplacements function for Skill replacement pattern. FAIL if any of the 3 body replacement functions lacks the pattern.

I14: No dead tool names in mappings

Grep tool mapping objects for known-dead names (e.g. Task). Regression guard — FAIL if dead tools reappear.

I15: Statusline VERSION detection

Verify scripts/gsp-statusline.js checks both current ({runtimeDir}/VERSION) and legacy ({runtimeDir}/get-shit-pretty/VERSION) paths.

I16: Skill execution_context refs resolve

Every @${CLAUDE_SKILL_DIR}/../../ reference in skill <execution_context> blocks must point to an existing source file. FAIL if any ref is broken.

I17: Skills with sibling files get full directory copy

If a skill directory contains files beyond SKILL.md (e.g. gsp-style/styles/), verify the installer's copy functions handle subdirectories (not just SKILL.md). FAIL if copy functions don't support siblings.

I18: Source skill dirs have gsp- prefix

All skill directories under gsp/skills/ (except get-shit-pretty) must start with gsp-. The installer copies as-is — no renaming. FAIL if any source skill dir is missing the prefix.

Step 5: Runtime Compatibility (R)

Read the baseline reference at ${CLAUDE_SKILL_DIR}/../gspdev-runtime-compat/references/baseline.md.

R1: Discovery paths match installer

For each runtime, check that the installer writes to the correct discovery paths:

• Claude: .claude/skills/, .claude/agents/
• OpenCode: .opencode/skills/, .opencode/agents/
• Gemini: .gemini/skills/, .gemini/agents/
• Codex: .agents/skills/ (NOT .codex/skills/), no agents

R2: Tool name mappings current (mapping + body)

Compare installer's tool mappings against baseline. Verify Skill replacement exists in body replacement functions. Any difference → WARN.

R3-R7: Body replacements, config paths, SKILL_DIR

Check the installer's body replacement functions cover all patterns from the baseline.

R8: Codex agents are skipped

Verify Codex runtime doesn't install agent .md files.

R9: Skill tool replacement uses lookahead guard

Grep each apply*BodyReplacements for (?= near Skill — ensures the replacement isn't a naive \bSkill\b that would corrupt "SKILL.md", "skills", etc. WARN if guard pattern is missing.

Step 6: Template Coherence (T)

T1-T7: Config fields, state templates, phase templates, exports index, chunk format, state/brief templates

See automated test suite for details.

Step 6b: Prompt Engineering Quality (P)

Automated checks for prompt engineering hygiene across skills and agents.

P1: Line count budgets

Flag files exceeding size thresholds: skills >300 lines, agents >150 lines.

P2: <rules> placement

<rules> sections should only appear in skills, not in agent definitions or system prompts.

P3: <rules> size budget

Flag <rules> sections exceeding 30 lines — large rules blocks dilute focus.

P4: Cross-file duplicate lines

Find non-trivial lines (>30 chars) appearing in 3+ files — signals copy-paste drift.

P5: Skill↔agent instruction overlap

When a skill spawns an agent, check for repeated instructions between them. Overlap means the agent gets the same instruction twice (from skill context + its own definition).

P6: Verbosity ratio

Ratio of constraint keywords (must, never, always, forbidden, required) + <rules> lines vs total content. Flag >40% — indicates over-prompting.

P7: Vague directive detection

Grep for known anti-patterns: "be natural", "use good tone", "write clean", "be helpful", "ensure quality", "be thorough", "be creative", "be professional", "use best practices". These are instructions the model can't act on consistently.

Step 7: Report

Output a terminal-formatted report:

GSP Integrity Audit
═══════════════════════════════════════

Version Sync
  ✅ V1. Version agreement .......... PASS (0.5.0)
  ⚠️  V2. CHANGELOG coverage ........ WARN

Contracts
  ✅ C3. Skill→agent refs ........... PASS
  ✅ C4. Agent tool validity ........ PASS
  ✅ C5. No orphan agents ........... PASS
  ✅ C6. Agent frontmatter .......... PASS
  ✅ C7. Skill frontmatter .......... PASS
  ✅ C8. Claude-only field set ...... PASS

Installer
  ✅ I1. Installer syntax ........... PASS
  ✅ I2. Skills exist ............... PASS (21)
  ✅ I3. Agents exist ............... PASS (11)
  ✅ I5. Bundle dirs ................ PASS
  ✅ I6. Files field ................ PASS
  ✅ I7. Codex skills path .......... PASS
  ✅ I8. Tool mapping objects ....... PASS
  ✅ I9. Conversion functions ....... PASS
  ✅ I10. Body replacers ............ PASS
  ✅ I11. Single-line field strip ... PASS
  ✅ I12. Multi-line block strip .... PASS
  ✅ I13. Skill tool rename ......... PASS
  ✅ I14. No dead tool names ........ PASS
  ✅ I15. Statusline VERSION ....... PASS
  ✅ I16. Skill refs resolve ....... PASS
  ✅ I17. Sibling file copy ........ PASS
  ✅ I18. Source dirs gsp- prefix ... PASS

Runtime Compatibility
  ✅ R1. Discovery paths ............ PASS
  ✅ R2. Tool name mappings ......... PASS
  ✅ R3-R7. Body replacements ....... PASS
  ✅ R8. Codex skips agents ......... PASS
  ✅ R9. Skill rename guard ......... PASS

Templates
  ✅ T1-T7. All template checks ..... PASS

Prompt Engineering
  ✅ P1. Line budgets ............... PASS
  ✅ P2. <rules> placement .......... PASS
  ✅ P3. <rules> size ............... PASS
  ✅ P4. Cross-file duplicates ...... PASS
  ✅ P5. Skill↔agent overlap ....... PASS
  ✅ P6. Verbosity ratio ............ PASS
  ✅ P7. Vague directives ........... PASS

─── Summary ───────────────────────────

  37 PASS · 1 WARN · 0 FAIL
  GSP pipeline is healthy with minor issues.

Important Notes

• Read-only — do NOT modify any files, only report findings
• Be specific — every issue names the exact file and suggests the exact fix
• Don't over-report — if the same root cause triggers multiple checks, note it once and cross-reference
• Count everything — the report should show exact counts (34 skills, 11 agents)
• Runtime compat uses baseline — read the baseline.md reference file, don't re-derive from scratch