cargo-crev-review

name	cargo-crev-review
description	Review Rust dependencies and create/publish cargo-crev package review proofs for the user. Use when the user asks you to review one of their Rust dependencies, audit a crate, or produce a crev proof.

cargo-crev: reviewing Rust dependencies as an agent

This skill tells you how to review a Rust dependency on the user's behalf and help them produce a signed cargo-crev package review proof that they can publish to their proof repository.

When to use this skill

Use this skill when the user asks you to:

• review one of their Rust dependencies (or a transitive dep),
• audit a specific crate version,
• produce a cargo crev review/proof for a crate,
• help them catch up on crates that still need reviewing.

Do not use this skill for unrelated security audits, non-Rust code, or trust proofs for other crev ids — those are separate workflows.

Prerequisites (check these first)

1. The user has a crev id configured:

   cargo crev id current
   

   If this fails, stop and ask the user to run cargo crev id new --github-username <them> first. Do not auto-generate an id.

2. The user's proof repo has been set up and is reachable. cargo crev repo dir prints the local path.

High-level workflow

1. Refresh the user's web of trust. Run cargo crev update once at the start of the session to pull the latest proofs from every trusted reviewer's proof repo (it's a shortcut for cargo crev repo update). Without this, both candidate discovery (which depends on the status column reflecting current trusted reviews) and the later cross-check against existing reviews can be working off stale data, causing you to either review a crate someone else just signed off on or miss a recent advisory. See "Refreshing the web of trust" below.
2. Pick the crate + version to review. If the user names a specific crate, skip to the next step. Otherwise, use the candidate discovery step below to produce a ranked list and confirm the pick with the user.
3. Start a review report file. See "The review report file" below. Every finding from every subsequent step — positive or negative — gets appended to this file as you go. This is your scratchpad, your audit trail, and the source you distill into the final comment.
4. Locate the crate's local source. Use cargo crev crate dir <name> <version> to print the path to the crate source tree that cargo downloaded from crates.io. Record the path in the report.
5. External verification. Check that what crates.io shipped matches the public upstream repository at a specific commit, and that that commit corresponds to the advertised version tag. See "External verification" below. Record the outcome — pass, partial, or fail — in the report. Any unexpected discrepancy is a real finding and must be raised to the user before proceeding.
6. Read the code. See "Reviewing the code" below. Append findings to the report as you go.
7. Cross-check against existing reviews. Query the local proof database for any existing reviews of this crate (any version), read them, and reconcile with your own findings. See "Cross-checking existing reviews" below. Record the comparison in the report.
8. Assemble the unsigned proof file. Fill in the YAML fields (rating, thoroughness, understanding, comment, issues, advisories, llm-agent). The comment is the full text of the report file by default — see "Assembling the unsigned proof" below.
9. Validate the proof file. Round-trip the unsigned proof through cargo crev review --import-unsigned-from ... --no-store --no-edit --print-unsigned. This step is mandatory — fix and re-validate until it passes. See "Round-trip validating the draft" below.
10. Hand off to the user. The agent does not sign or publish. Give the user two files — the report and the unsigned proof — along with the exact command they should run to interactively review, edit and sign it. See "Handing off to the user" below.

The review report file

Maintain a single markdown file per review, at:

target/crev/reviews/<crate>-<version>.md

(Create the directories if they don't exist.) Start it as soon as a candidate is picked, before any inspection begins. Append to it throughout the review — treat it as an append-only journal.

Minimum structure:

# Review: <crate> <version>

- Local source: <path from `crate dir`>
- Upstream repository: <URL or "unknown">
- Upstream commit verified: <sha or "n/a">
- Upstream tag for version: <tag or "n/a">
- Verification outcome: pass | partial | fail

## External verification

<details of what was checked, what matched, what differed>

## Code review findings

<ongoing notes — one bullet per observation, grouped by file or theme>

## Cross-check against existing reviews

<how many prior reviews, agreement, disagreement, verdict on whether
they change your draft fields>

## Open questions / things skipped

<anything you didn't examine and why>

## Draft review fields

- rating: ...
- thoroughness: ...
- understanding: ...

Rules:

• Append, don't rewrite. If you change your mind about something, add a follow-up note; don't quietly edit earlier entries. The user should be able to see how your understanding evolved.
• Every problem, however small, goes in. Even if you decide it's not worth reporting in the final proof, it belongs in the report so the user can sanity-check your judgement.
• Record what you did not look at. If you skipped the test directory, or only skimmed build.rs, say so explicitly under "Open questions / things skipped". This is what backs honest thoroughness / understanding values later.
• Hand the report to the user alongside the unsigned proof file at the end of the workflow (see "Handing off to the user"). The report is how the user actually audits your work; the unsigned proof is the thing they sign after auditing.

Refreshing the web of trust

Before any candidate discovery or cross-checking, pull the latest proofs from every trusted reviewer:

cargo crev update

This is a shortcut for cargo crev repo update. It walks every proof repository in the user's WoT and git fetches it, so the local proof database reflects the current state of the world. It can take a few seconds to a minute depending on how many ids the user trusts and network conditions; that's normal.

Run this once per session, before the first candidate-discovery pass, and again only if the session has been running long enough that the data is plausibly stale or the user explicitly asks. It is not necessary to re-run it between successive reviews in the same session.

If the command fails for some repos (network blip, dead remote, auth issue) but succeeds for others, that is usually fine — note the failures in case they matter, but proceed. If it fails entirely (e.g. no trusted ids configured, so there is nothing to fetch), stop and tell the user: candidate discovery and cross-checking will both be meaningless until the WoT is set up.

Finding candidate crates to review

When the user hasn't named a specific crate, first make sure the WoT is fresh (see "Refreshing the web of trust" above), then capture a stats dump of their dependencies.

cargo crev verify is slow — it walks every dependency, downloads metadata from crates.io, and computes trust graphs. On a non-trivial project it can take 30 seconds to several minutes. Therefore:

• Run it exactly once per session and save the output to a file.
• Never re-run it unless the user explicitly asks or the dependency tree has changed (e.g. after cargo update).
• All candidate discovery, sorting, and filtering works on the saved file — re-read the file instead of re-running the command.

cargo crev verify \
    --show-reviews true --show-downloads true \
    --show-loc true --show-flags true \
    --force-print-header \
    > target/crev-verify.txt 2>&1

Notes:

• Only request the columns you actually need for candidate selection: reviews, downloads, loc, and flags. The default output already includes status, name, and version.
• Do not use --show-all — it enables many extra columns (geiger, owners, issues, lpidx, latest trusted version, …) that produce wide, hard-to-parse output and are not needed for picking candidates. If you need a specific extra column later (e.g. --show-geiger true for a particular analysis), add it to a one-off query for that crate, not to the full dump.
• --force-print-header makes the tool print the column header even when stdout is redirected. Always use this when capturing — the header is the only reliable way to know which column is which.
• Exit status 255 is expected and does not mean the command failed. It signals VerificationFailed — i.e. at least one dependency is not yet fully verified, which is obviously the case when you're about to start reviewing. Only treat it as a real error if the stdout file is empty or the stderr contains an actual error message.
• Status column values you'll see:
  • local — crate from a local path source (skip; not reviewable via crates.io).
  • none — no existing trusted review; prime candidate.
  • pass — already has sufficient trusted reviews; skip.
  • flagged / dangerous — has issues reported against it; these may be worth confirming rather than reviewing fresh.
  • N/A — there are no trusted ids in the user's WoT at all. If you see this, stop and tell the user they should set up trust first (cargo crev trust <id>) before candidate discovery is meaningful.

Once the file exists, work from it for all subsequent candidate selection in this session. If you need to re-examine the data, read target/crev-verify.txt — do not re-run cargo crev verify.

Before narrowing down, confirm the criteria with the user — see the "Picking candidates" section below.

Picking candidates

Picking what to review is inherently a bit arbitrary, but the default goal is: find the non-passing, non-local, least-reviewed, least-downloaded package — i.e. the most obscure crate in the user's dependency tree that nobody else is likely to be reviewing any time soon. Unless the user says otherwise, apply these filters/rankings in order:

1. Filter by status. Keep only status = none. Skip pass (already covered), local (not from crates.io), and N/A (configuration problem, handled separately). flagged / dangerous are a separate workflow (confirming pre-existing issue reports) — don't pick them unless the user asked.
2. Sort by lowest total review count. Use the second reviews column (total across all versions, not the per-version count). Zero-review crates come first.
3. Tiebreak by lowest total download count. Popular crates will eventually get reviewed by someone else; obscure ones won't. Reviewing the long tail is higher marginal value. Use the second downloads column (total).
4. Tiebreak by CB flag. Among otherwise equal candidates, prefer crates with a custom build script — they run arbitrary code at build time and are higher-risk, so the review is more valuable.

If the user has additional or different preferences (e.g. "I only care about direct deps", "prioritize high lpidx", "skip -sys crates"), they override the defaults above — ask if unsure.

The user's own crates are valid candidates. If a top candidate turns out to be authored by the user themselves, don't filter it out — a review where the author says "I wrote this and had an LLM agent audit it" is still a useful signal for downstream consumers, and once published it'll flip to status = pass and drop out of the filter naturally. Agent provenance is recorded structurally via the llm-agent: field on the proof (see "Disclosing agent provenance" below), so no special comment wording is needed for own-crate reviews.

Pick one candidate, not a shortlist. Filter the captured stats file by the rules above, sort by the priority order (status → review count → download count → CB flag), and propose the single most promising crate to the user. Don't offer a menu — it just slows things down. After that crate is reviewed and published, repeat the selection process on the same captured file (or re-capture if it's stale) to pick the next one.

Column legend

These are the columns present in the recommended cargo crev verify output (with --show-reviews, --show-downloads, --show-loc, --show-flags). Run cargo crev verify --help for the full legend including columns not shown by default.

- status     - Trust check result: `pass` for trusted, `none` for lacking reviews,
               `flagged` or `dangerous` for crates with problem reports.
               `N/A` when crev is not configured yet.
- reviews    - Number of reviews for the specific version and for all available
               versions (total). Two sub-columns: version count / total count.
- downloads  - Download counts from crates.io (version / total).
               Two sub-columns: version count / total count.
- loc        - Lines of Rust code
- flgs       - Flags for specific types of packages
  - CB         - Custom Build (runs arbitrary code at build time)
  - UM         - Unmaintained crate
- crate      - Crate name
- version    - Crate version

External verification

Before reading a single line of the crate's code, verify that what crates.io shipped actually matches the public upstream repository. This catches the single highest-leverage class of attack (a publisher whose account was compromised, who can ship a tarball that doesn't match their public git history). It also tells you which upstream commit the reviewed snapshot corresponds to, which is valuable information in its own right.

Ideal outcome, recorded in the report file:

Package matches public repo <url> at commit <sha>, which is the commit tagged as <tag> corresponding to version <version>.

Anything short of that is a partial or failed verification, and every discrepancy must be recorded in the report and raised to the user. Don't silently proceed past a verification issue — if the user hasn't told you what to do in that case, stop and ask.

Procedure

1. Get the local source path.

   cargo crev crate dir <name> <version>
   

   Record it in the report. Call this path $SRC.

2. Look for .cargo_vcs_info.json at $SRC/.cargo_vcs_info.json. When present (it usually is), it contains the upstream git sha the tarball was built from:

   { "git": { "sha1": "abc123…" }, "path_in_vcs": "subcrate" }
   

   Treat this as a hint, not a guarantee. The file may be missing (older crates, unusual publish flows), or the sha may be slightly off (dirty worktree at publish time). If it's missing, note that in the report and fall back to searching the upstream repo by version tag instead.

3. Find the upstream URL. Read $SRC/Cargo.toml.orig (the pristine pre-publish manifest) and extract package.repository. If absent or dead, also check package.homepage, then crates.io metadata. If none of those point at a reachable repository, search the web (e.g. "<crate-name>" site:github.com or the author name plus the crate name) — crates sometimes move or get renamed and the manifest stays stale. Record every URL you tried and which one resolved. If you still cannot find a public source after a reasonable search, record the failure and ask the user how to proceed. Do not skip external verification silently.

4. Clone (or update) the upstream cache. Use a stable cache directory so repeat runs don't re-clone:

   mkdir -p target/crev/review-cache
   UPSTREAM=target/crev/review-cache/<crate>
   if [ -d "$UPSTREAM" ]; then
     git -C "$UPSTREAM" fetch --tags origin
   else
     git clone --filter=blob:none "$REPO_URL" "$UPSTREAM"
   fi
   

   --filter=blob:none keeps the clone fast even for huge histories — blobs are fetched on demand when you actually diff.

5. Identify the upstream commit to compare against. Prefer, in order:

   1. The sha from .cargo_vcs_info.json (if it exists upstream).
   2. The commit that the version tag points to — tag naming conventions to try: v<version>, <version>, <crate>-v<version>, <crate>-<version>.
   3. If neither matches, report the failure and ask the user. Record in the report which method succeeded and which commit/tag was chosen. Ideally both the sha and the version tag agree — if they disagree (e.g. vcs_info points at a commit the tag does not), that is itself a finding and must be reported.

6. Check out the commit.

   git -C "$UPSTREAM" checkout "$SHA"
   

7. Compare the trees. Compare $SRC against $UPSTREAM/<path_in_vcs> (or $UPSTREAM if path_in_vcs is empty or .). Use diff -r --brief for an initial pass, then drill into any differing file with a full diff to see the content.

   Files expected to differ or to exist on only one side — add these to an ignore list and do not flag them:

   • Cargo.toml — always rewritten by cargo publish. Compare $SRC/Cargo.toml.orig against upstream's Cargo.toml instead. Workspace-inherited fields may differ; those differences are expected and benign as long as they're clearly inheritance artefacts.
   • Cargo.toml.orig — exists only in the crate tarball.
   • .cargo_vcs_info.json — exists only in the crate tarball.
   • .cargo-ok — cargo extraction marker.
   • Cargo.lock — sometimes present, sometimes stripped.
   • .gitignore, .gitattributes — often excluded via exclude.
   • target/ — build artifacts, never in the tarball.

   Any other difference is real and must be recorded in the report with the file path and a short description of what differs.

8. Cross-check the manifest. Diff $SRC/Cargo.toml.orig against $UPSTREAM/<path_in_vcs>/Cargo.toml at the checked-out sha. Expected differences for workspace-published crates: fields like version, authors, edition, license being inlined from the workspace root; [workspace] stanza absent from the crate copy; path dependencies resolved to version specs. Flag anything else.

9. Record the outcome in the report under "## External verification", e.g.:

   Package matches public repo https://github.com/foo/bar at commit abcdef0123 tagged v1.2.3. Manifest differences are limited to expected workspace-inheritance artefacts (listed above). All other files bit-for-bit identical.

   or, on failure:

   Package does not cleanly match upstream. The file src/util.rs differs from the upstream copy at the commit recorded in .cargo_vcs_info.json (see diff below). Raising to user before continuing.

Any verification failure blocks the rest of the review until the user says what to do.

Internal verification (reviewing the code)

Once external verification has passed (or the user has explicitly told you to proceed past a failure), start the actual code review. This is "internal verification": convincing yourself that the code inside the verified tarball does what it claims to and nothing more.

Mindset

Approach the code adversarially. Assume, as a working hypothesis, that everything you are reading could be wrong or hostile, and it is your job to either falsify that hypothesis or document what you found. This is not paranoia; it's the baseline for a review that is worth signing your name to.

Concrete rules that follow from this: