| name | agent-ready-cli-audit |
| description | Audit an existing CLI against the Agent-Ready CLI Checklist. Evidence-first: identifies the target command, runs it with --help and safe read-only commands, finds install instructions and the docs page, scores the checklist, and produces prioritized fixes. Use when user says '/agent-ready-cli-audit' or asks to audit a CLI, check whether a CLI is agent-ready, or review a CLI/repo/docs for agent readiness — including questions like 'is our CLI good for agents?' or 'why do coding agents get stuck with our tool?'. Does not design new commands. |
| license | MIT |
| metadata | {"version":"0.6.0","author":"Emmanuel Paraskakis / Level 250"} |
Agent-Ready CLI Audit
Overview
Use this skill to evaluate an existing CLI against the Agent-Ready CLI Checklist.
Audit mode is evidence-first. Inspect docs/code and run the CLI where possible. Do not replace missing evidence with imagined best practices.
The audit asks:
Would a coding agent get stuck trying to discover, authenticate, inspect, plan, act, verify, and recover?
Included References
references/agent-ready-cli-checklist-v2.md — canonical checklist and scoring rubric. Use it as the source of truth for scoring.
Scope and Routing
Triggering is defined in the frontmatter description. In scope: evidence-based review of an existing CLI, repo, or docs. Out of scope: greenfield command design (use agent-ready-cli-spec) and implementation (use agent-ready-cli-build).
Inputs
| Input | Required | Notes |
|---|
| Target CLI | Yes | Installed command name, repo path, npm/PyPI/Homebrew package name, or docs URL — any one is enough. |
| Focus workflows | No | Specific workflows the user cares about most. |
| Safe environment | No | Whether mutating commands may be exercised. Default: no — read-only commands only. |
| Credentials | No | Env var names for auth'd commands. Never ask the user to paste secrets into the conversation; ask for names of env vars already set in the shell. |
The target can arrive as a command name, a path on disk, a package name, a docs URL, or a GitHub repo. Never go looking on disk and report what you found there.
Gather inputs once, then run unattended. If the target CLI is named in the request, skip questions entirely — proceed straight to Step 0 with the table's defaults for everything else.
Only if the target is missing, ask once, like a person. Do not announce that a directory is empty, do not cite these instructions:
Which CLI should I look at? A command name, a repo path, a package name, or a docs URL — whatever you've got.
Also: is it safe to run commands that change things, or should I stick to read-only? (Read-only is my default.)
After that, complete the entire audit without further questions. When anything is ambiguous mid-run, choose the safe option (read-only), log the assumption in the report, and continue.
Expected fallbacks (all fine — log them, lower confidence where noted):
- Source/tests not reasonably available (closed or large repo): fail category 14's boxes you could not verify, mark the audit low-confidence, and say so — do not clone everything, and do not reclassify them as N/A.
- No web access: skip the docs-page fetch, note degraded confidence on docs-related items.
- Clean-environment install can't be truly verified read-only: that item is (not scored), so it costs nothing. Report what package metadata and docs imply, as an observation.
- Working directory not writable: save the report wherever writes are permitted and state the path.
- Environment already authenticated as a real user: read-only commands are still fine; cite only the minimum account evidence needed (no token values, no scopes dumps beyond the finding).
Audit Workflow
0. Identify and onboard the target
Before scoring anything, onboard the way a fresh agent would:
- Locate the command:
which <tool>. If not installed, find install instructions (README, npm view <pkg>, brew info <tool>, PyPI page). Do not install without the user's permission; audit from docs/source with lowered confidence instead.
- Run discovery commands:
<tool> --help, <tool> -h, <tool> --version, and --help on the key subcommands.
- Find the install path: identify the documented one-line install (brew, npm/npx, pipx/uvx, curl script, releases page) and whether it would work in a clean environment.
- Find the docs page: from help-text links, README links, or package metadata. If web access is available, load the docs page and note whether it shows the top workflows as copy-paste commands.
Completion criterion: the audit records where the CLI comes from, how it installs, and where its docs live — or explicitly states these could not be found (that is itself a finding).
1. Gather evidence
Inspect, as available:
- README/docs/help pages;
- package metadata and executable entrypoint;
- source code for command parsing, I/O, auth, exit codes;
- tests;
- installed CLI behavior.
Run safe read-only commands where possible:
tool --help
tool --version
tool auth status --json
tool <resource> --help
tool <resource> list --json
Avoid destructive commands unless the user explicitly provides a safe test environment.
Stopping rule: don't --help every leaf of a large CLI. Sample until each checklist category has at least one piece of direct evidence, favoring the user's focus workflows; then stop.
Run the agent test. It is the checklist's central experiment, not an appendix — see "The agent test" in references/agent-ready-cli-checklist-v2.md. Drive the user's focus workflow yourself through discover → auth → inspect → plan → act → verify, using read-only commands, or a mutation behind --dry-run where the environment is safe. Record the transcript.
The test scores no box of its own — "Agent eval is run before claiming agent-readiness" is a claim about the team's process and is (not scored). Run it anyway: it is the cheapest way to gather evidence for the [C] gates, which nothing else exercises end to end. A gate you never drove is a gate you cannot pass.
When the test genuinely cannot be run — the CLI is not installed, no safe environment, no credentials — say so, drop Confidence, and note in the Verdict that the gates were assessed from source rather than from behaviour.
Completion criterion: audit distinguishes observed evidence from untested assumptions, and either includes an agent-test transcript or states why none exists.
2. Score the checklist
Score per references/agent-ready-cli-checklist-v2.md, under "How to score". Follow it; do not invent a scheme.
Every box is worth one point. Score = passed ÷ scored, as a percentage. 130 boxes; 4 are (not scored) and never counted; 126 scorable before N/A.
- A box passes only on evidence you gathered — a command you ran, a file you read. A claim in the README passes nothing.
- N/A leaves the denominator; unverified does not. N/A means the item cannot apply to this product. It never means "I could not check it." An item you could not verify fails, costs the box, and lowers Confidence. N/A is the only lever that moves the denominator — justify every use of it, per item.
- The 9 boxes marked [C] are gates. They score one point like any other, but failing any one caps the Verdict at Partially ready, whatever the percentage. So does an agent-hostile category, which additionally scores zero across all its boxes.
The percentage is completion; the Verdict is agent-readiness. They are different claims. A polished human CLI can tick most of 130 boxes and still fail every gate, because the things that break an agent's loop are a small share of the count. Never let a high percentage imply an agent can drive the tool — that is what the gates are for. Report both, and say which gates failed.
Because N/A varies per CLI, denominators vary. Compare percentages across tools, never raw counts.
Categories:
- Product-surface fit
- Discoverability
- Predictable command grammar
- Noninteractive automation
- Machine-readable I/O
- Composability and stdin
- Authentication and environment
- Inspectability and verification
- Safe side effects
- Errors, recovery, and exit codes
- Versioning and updates
- Distribution and runtime
- Security/privacy/enterprise trust
- Testing and release quality
- Agent onboarding package
Score against the checklist, never against another CLI. If you have recently seen a sibling tool that went beyond the rubric, that extra feature is not the bar. Example: for a plain API-key scheme the checklist (category 7) says documented key acquisition plus auth status satisfies the item — a set-token command is a convenience, and its absence costs nothing.
Pre-distribution targets get two scores. When the target is confirmed local-only — the user says so, or no install path resolves and no registry, tap, or release exists — categories 11 (updates) and 12 (distribution) measure a stage the product has not reached, not a defect in it. Report both numbers:
- Raw score. Passed ÷ scored, honestly, on real evidence. Nothing is inflated.
- Agent-readiness score, with categories 11 and 12 removed from the denominator — 17 scorable boxes, leaving 109 before any other N/A — and a one-line note saying which categories were deferred and why.
Drive the Verdict from the agent-readiness score when the target is pre-distribution; otherwise from the raw score. Never silently drop the raw number, and never mark 11/12 as N/A — "not shipped yet" is a fact about today, while N/A is a fact about the product forever.
Completion criterion: each score has evidence and a fix.
3. Identify agent blockers
Prioritize blockers that prevent the execution loop:
- no noninteractive auth;
- no inspect/status commands;
- no JSON output;
- stdout/stderr mixed;
- no dry-run/confirm for mutations;
- no verification command;
- bad exit codes;
- hangs without TTY;
- undocumented install/update path;
- secret leakage or unsafe shell-outs.
Completion criterion: top 5 fixes are ordered by agent-blocking impact.
4. Recommend next path
Depending on score:
- strong CLI: recommend agent eval and docs/Skill polish;
- partial CLI: recommend spec patch and implementation roadmap;
- human-only CLI: recommend headless layer beneath prompts/TUI;
- wrong surface: recommend API/MCP/Skill/UI instead of CLI.
Completion criterion: recommendation is actionable and does not overclaim certainty.
Output Format
Save the full report to a file (default agent-ready-cli-audit-<tool>.md in the working directory, or wherever the user asked) and give the verdict summary in the conversation.
# Agent-Ready CLI Audit: [CLI]
## Verdict
Status: Agent-ready / Mostly ready / Partially ready / Human-only CLI / Wrong surface
Score: N/M scored items = P% [M = 126 minus N/A; state the N/A count]
Agent-readiness: N/M = P% [omit unless pre-distribution; name the deferred categories and why]
Critical gates: N/9 passed — [name every failed [C] gate; any failure caps the Verdict at Partially ready]
Not scored: 4 process/future claims, reported below as observations, never as points
Stage: published / pre-distribution (local-only)
Confidence: high/medium/low based on evidence available
Independence: independent audit / self-audit of a CLI this same run built [see pitfall 9]
> The percentage is checklist completion, not a safety measure. It does not mean an agent
> will succeed P% of the time, and the missing share is not necessarily minor.
## Target
Command: ... | Version: ... | Install path: ... | Docs page: ...
## Assumptions
- [anything assumed because the run was unattended]
## Evidence inspected
- ...
## Scorecard
| Category | Score | Evidence | Recommended fix |
|---|---:|---|---|
## Agent blockers
1. ...
## Quick wins
- ...
## Roadmap
### High priority
### Medium priority
### Low priority
## Agent eval prompt
[prompt tailored to this CLI]
Common Pitfalls
- Auditing from docs only when the CLI can be run. Run safe commands where possible.
- Penalizing non-applicable items. Mark N/A honestly if Docker/plugin/etc. is irrelevant.
- Overvaluing pretty terminal UX. Rich output is fine only if headless/JSON mode works.
- Ignoring tests. Agent-readiness should be testable, not just documented.
- Inventing command contracts during audit. Audit first; spec redesign second.
- Interrogating the user. One consolidated question round at most; then run to completion.
- Reading a low 11/12 as brokenness. An unpublished CLI legitimately scores low on updates and distribution. Score it honestly, then report the agent-readiness score alongside, and say in the Verdict how many lost points are stage-expected versus real gaps.
- Trusting a test's name over what it asserts. "23/23 passing" and "the tests cover what their names claim" are different facts. Before crediting a category on test evidence, open at least one test and read what it actually asserts. A test called "
--json is valid on success and error paths" that never passes --json proves nothing.
- Auditing your own build without saying so. When the same run that built the CLI also scores it, the audit is not independent — it inherits the build's blind spots and tends to score high. Re-derive every claim by executing commands, never by trusting the build phase's own summary. State plainly in the Verdict that this is a self-audit, and recommend an independent
agent-ready-cli-audit run before the score is quoted anywhere.
- Scoring against a sibling CLI instead of the rubric. Another tool exceeding the checklist does not raise the bar. Cite the checklist item, not the comparison.
Verification Checklist
