skill-portfolio-auditor

name

skill-portfolio-auditor

description

Audit Codex and compatible filesystem skill portfolios for exact or partial duplication, unclear boundaries, stale variants, and orchestrator-worker opportunities. Use when a user wants to inventory one or more provider skill directories, compare changes, review overlapping skills, maintain an evidence-backed skills wiki, or prepare safe consolidation proposals without automatically modifying runtime skills.

Skill Portfolio Auditor

Audit a local skill portfolio as an evidence-first, read-only-by-default orchestrator. The goal is portfolio clarity, not the smallest possible skill count.

Inputs

• A skills root, normally ${CODEX_HOME:-$HOME/.codex}/skills.
• A workbench outside the scanned root.
• Optional previous inventory manifest for delta mode.
• Optional wiki root outside the scanned root.
• Audit mode: daily, weekly, or manual.
• Provider adapter: codex, claude-code, gemini, antigravity, github-copilot, qwen, kimi, perplexity, zai, glm, or custom-filesystem.

Outputs

• Normalized inventory manifest and source hashes.
• Added, changed, moved, removed, and unchanged skill sets.
• Candidate overlap pairs and clusters with multidimensional evidence.
• Boundary and architecture packets.
• Independently verified advisory proposals.
• Optional sanitized wiki staging projection.
• Human-owned Decision Records for accepted decisions.

Non-goals

• Never merge, delete, rename, install, or rewrite runtime skills automatically.
• Never classify duplication from names, descriptions, or one score alone.
• Never treat worker completion as acceptance.
• Never put the wiki or workbench inside the scanned root.
• Never copy secrets, raw private prompts, cookies, tokens, or private traces into reports or a public package.

Workflow

1. Resolve the provider root with scripts/provider_adapter.py. Read references/provider-adapters.md and preserve its support tier.
2. Pin the scan, workbench, and optional wiki roots. Reject self-scanning layouts and symlink escapes.
3. Run scripts/inventory_delta.py inventory to build a fail-closed manifest. Read references/inventory-delta-contract.md.
4. For a previous accepted manifest, run delta comparison and limit daily analysis to changed identities plus registry context.
5. Run scripts/analysis_pipeline.py with an explicit pair budget. Read references/analysis-worker-contract.md.
6. Treat candidate scores as routing signals only. Compare triggers, inputs, outputs, workflow, tools, evidence, risks, and journey position.
7. Run scripts/verification_worker.py against the exact analysis hash. The producer cannot approve its own output.
8. Write only sanitized, allowlisted pages to wiki _staging through scripts/wiki_sync.py. Read references/wiki-sync-contract.md.
9. Present proposals to the human owner. Record accepted decisions with ID, date, source hashes, decision, and rationale.
10. If a mutation is approved, create a separate migration task with backup, allowlisted writes, old-vs-new evals, independent review, exact post hashes, and verified rollback.

Orchestrator and worker boundaries

• Inventory/delta worker owns filesystem facts, not architecture decisions.
• Similarity analysis finds candidates, not duplicates.
• Boundary analysis distinguishes justified specialization, sequential stages, shared workers, and true overlap.
• Quality analysis reports observable maturity signals and unavailable evidence.
• Architecture analysis proposes keep-separate, group, orchestrator-worker, merge-evaluate, or defer.
• Verification independently accepts, rejects, or returns evidence to a worker.
• Wiki sync projects accepted evidence; it does not create decisions.
• Provider adapters select compatible roots and report support tiers; they do not claim provider runtime behavior or modify provider state.
• The human owner is the only authority for consolidation and runtime mutation.

Evidence and acceptance

Every recommendation must cite source hashes and inspected dimensions. Missing evidence yields defer, never a confident duplicate label. Keep completion, artifact existence, verification, orchestrator acceptance, and human decision as separate states.

Failure modes and retries

Read the closed taxonomy and retry policy in references/failure-modes.md.

• tool_unavailable: verify once, report fallback or block.
• permission_blocked: stop until explicit approval.
• timeout: retry once with narrower scope.
• partial_output: one forward-fix, then return to review.
• schema_mismatch: at most two corrections.
• security_blocked: reject immediately; do not retry unchanged input.
• regression_detected: reopen the responsible stage and preserve evidence.

Do not exceed bounded retries or silently expand scan/write scope.

Tool verification

The deterministic core requires Python 3.11+ and its standard library. Verify python3 --version before execution. Filesystem read/write capability is required for the selected scan, workbench, and wiki paths. Missing tools produce tool_unavailable; do not install packages automatically. See references/production-skill-architecture.md.

Verification

Run from the skill root:

python3 -m unittest discover -s scripts -p 'test_*.py'
python3 scripts/validate_behavioral_evals.py evals/evals.json
python3 scripts/provider_adapter.py --provider codex

Before installation or publication, also run the skill-creator-v2 lint, quick validation, sanitation, all-checks, package, and dry-run install gates.

Final review

Before installation or release, run a separated final review covering task completion, trigger clarity, tool reliability, robustness, and first-time user experience. Require at least 9/10, preserve the machine-readable review artifact, and allow at most two revision cycles. A built package is not accepted without sanitation and install-smoke evidence.