cli-release-audit

name	cli-release-audit
description	Pre-release audit for CLI tools (especially Rust/Python). Checks build, tests, help quality, output friendliness (agent + human), path leaks, dependency portability, .gitignore hygiene, and missing release artifacts.
version	1.0.0
metadata	{"hermes":{"tags":["release","audit","cli","rust","qa","pre-release"],"related_skills":["dogfood","requesting-code-review"]}}

CLI Release Audit

Systematic pre-release review for CLI projects. Trigger when a user says "about to release", "publish", "pre-release check", or asks to review a CLI project for release readiness.

Audit phases — execute in order

Phase 1: Build & test health

1. cargo check / equivalent build command — must pass clean
2. Run full test suite (cargo test). Any failure is P0.
3. Note warning count — clippy warnings are P2 but worth flagging.

Phase 2: Dependency portability

1. Check Cargo.toml (or equivalent) for path = "..." local dependencies — these break on other machines. P0.
2. Check for hardcoded absolute paths in source: rg "/Users/\|/home/\|C:\\\\Users" src/
3. Check for hardcoded localhost URLs that aren't configurable defaults.

Phase 3: CLI help quality

1. Run --help for every subcommand.
2. Check: do all arguments have description text? Bare --symbol <SYMBOL> with no help doc is P1.
3. Check: is there a --version flag? Missing = P2.
4. Check: are default values shown and sensible?

Phase 4: Trial runs

1. Run with minimal/no data to check error messages — are they clear and actionable?
2. Run with existing state (if any) to see real output.
3. Check: does the tool panic or give a clean error on missing input?

Phase 5: Output friendliness

For agents:

• Is there a compact/machine-readable output mode?
• Is the JSON output flat enough to extract key fields without deep nesting?
• Are there redundant/repeated fields inflating token cost?
• Is there a --compact or --agent flag? If not, flag as P1-P2.

For humans:

• Is there a human-readable summary mode (not just raw JSON)?
• Are key decisions (go/no-go, direction, quality) visible without scrolling?
• Are internal IDs, policy hashes, and debug fields hidden by default?
• Is decision_hint or equivalent readable, or is it a packed internal string?

Phase 6: Information leakage

1. Check output for local absolute paths (user home dirs, download folders).
2. Check for leaked internal state paths in recommended commands.
3. Check for PII or machine-specific info in default output.

Phase 7: Repository hygiene

1. .gitignore — are runtime state dirs, __pycache__/, temp files excluded?
2. Are state/experiment directories already tracked? (git ls-files 'state*' etc.)
3. Is there a LICENSE file?
4. Cargo.toml / package.json — are repository, license, authors fields present?
5. Are there .DS_Store files tracked?

Phase 8: Documentation

1. Does README cover: install, first run, common workflows?
2. Is there example data or a --demo mode for new users?
3. Are internal dev docs separated from public docs?

Severity guide

Level	Meaning	Examples
P0	Blocks release — other machines can't build/run	Local path deps, test failures
P1	Serious UX/trust issue	No help text, path leaks in output, no human-readable mode
P2	Polish	Missing LICENSE, clippy warnings, no --version, bloated main file

Pitfalls learned

• Array-before-object check ordering: if code does as_object() then is_array(), the array case is unreachable. Check array first.

• .gitignore patterns like /state don't match state_foo/ — need state*/ glob.

• Even after adding /state*/, a root-level runtime file like state_autoresearch_cycle_validation.next-spec.json will still remain as D + ?? after git rm --cached unless it is separately ignored. For release cleanup, verify both tracked directories and similarly named root files with git ls-files --stage -- <path> plus git status --ignored --untracked-files=all.

• If a runtime artifact lives at repo root (for example state_autoresearch_cycle_validation.next-spec.json), state*/ still will not match it. Add an explicit root ignore like /state_autoresearch_cycle_validation.next-spec.json, then git rm --cached -- <file>. Expect a staged D until commit; in git status --ignored the working-copy replacement should appear as !!, which is the correct post-cleanup state.

• If a runtime artifact exists both as a tracked repo-root file and as ignored state directories, git rm --cached may leave the expected mixed state (D tracked_file plus ?? tracked_file). In that case add an explicit root ignore entry (for example /state_autoresearch_cycle_validation.next-spec.json), verify with git check-ignore -v --no-index, and treat the final clean state as: normal git status shows only the staged deletion while git status --ignored shows !! tracked_file.

• serde_json::to_string_pretty for all output means no compact mode exists by default.

• Recommended-next-command fields that embed absolute data paths from the developer's machine are a common leak vector.

• If you add --output-format / --compact / --agent / --human, verify the flags are actually wired through every output path — not just the default branch. In -style commands this includes: full snapshot output, filtered views (, , , hard-block filters), and named phase surfaces.