ci-code-review

name

ci-code-review

description

Deep code review of a pull request using parallel analysis agents (semantic consistency, bugs, tech debt, security). USE FOR: - Reviewing PRs for bugs, security issues, and code quality - Analyzing new abstractions for consistency and correctness - Identifying tech debt and architectural concerns - Posting review comments to specific lines on GitHub TRIGGERS: - "review PR", "code review", "review changes" - "diff review", "PR feedback", "check PR" - "analyze diff", "critique code", "review code" - "pull request review", "GitHub PR review" Use when this capability is needed.

Provide a code review for the given pull request.

Follow these steps:

1. Eligibility Check (Sonnet): Check if the PR (a) is closed, (b) is automated/trivial. If so, stop.

2. PR Analysis (Fable): View the PR and return:

   • Summary of the change and its purpose
   • List of new functions, types, enums, or abstractions introduced
   • For each new abstraction: its name, stated purpose (from comments/docs), and intended usage contract

3. Parallel Deep Review (4 Fable agents):

   Pass the PR summary and new abstractions list to each agent.

   a. Semantic Consistency Agent: For each new function/type/enum introduced:

   • Read its definition, documentation, and any comments describing when/how it should be used
   • Find ALL usages of that abstraction within the PR
   • Verify each usage matches the documented intent
   • Flag misuse: e.g., error-handling functions called for wrong error types, validation functions bypassed, enums used inconsistently
   • Pay special attention to: panic/error functions (when should they trigger?), unsafe blocks, security-sensitive operations

   b. Deep Bug Analysis Agent: Read the full context of modified files (not just diff lines).

   • Understand the data flow and control flow around changes
   • Check for logic errors, edge cases, off-by-one errors, resource leaks
   • Verify error handling is appropriate for each failure mode
   • Check that invariants are maintained across the changes

   c. Tech Debt Removal Agent:

   • Understand new abstractions that are introduced, new functions/types/enums
   • Identify possible future usecases for these things and understand whether abstractions meet future requirements
   • See which paradigms of Rust (or other language) development are used and whether they apply here
   • Identify possible improvements that would benefit long term maintainability of the code
   • Be an enjoyer of abstractions: generics, traits, dyn, enums, etc.

   d. Security Reviewer Agent:

   • Identify whether changes to the protocol do not break soundness
   • Identify possible attack vectors that are introduced with these changes
   • Check for input validation gaps at trust boundaries (user input, network data, file I/O, IPC)
   • Verify authentication/authorization checks are not bypassed or weakened
   • Look for injection risks: SQL, command, path traversal, template injection, deserialization
   • Check cryptographic usage: hardcoded secrets, weak algorithms, nonce reuse, timing side-channels
   • Verify resource limits: unbounded allocations, missing timeouts, denial-of-service vectors
   • Check concurrency: TOCTOU races, lock ordering, shared mutable state without synchronization

4. Validate Issues (MANDATORY — do not skip):

   After collecting all issues from the 4 agents, validate every issue scored >= 50.

   For each issue from the agents:

   • For complex logic/semantic issues, reason through whether the bug is real and exploitable.
   • For issues you can verify mechanically (e.g., a failing test), prefer direct verification (run the test).

   Score each issue 0-100 AFTER validation:

   • 0: False positive, doesn't stand up to scrutiny, or pre-existing issue
   • 25: Might be real, but couldn't verify. Stylistic issues without explicit guidance.
   • 50: Verified real issue, but minor/nitpick. Not important relative to PR scope.
   • 75: Verified real issue that will impact functionality. Insufficient existing approach.
   • 100: Confirmed real issue that will happen frequently. Direct evidence confirms it.

5. Post comments to PR: Always post a single review to the PR so the author has confirmation that the review ran, regardless of whether issues were found.

   • When there are validated issues with score >= 50, include them in the comments array.
   • When there are none, post the review with an empty comments array and a body that briefly states no issues were found (1-2 sentences, same tone as the comment guidelines below — concise, senior-engineer voice, no scores/severity labels/ceremony).

   Runs in CI — do not pause, do not ask for user confirmation, do not list issues for approval.

   Build a JSON file and use the GitHub review API:

   # Write review JSON to a unique temp file (use $$ for PID to avoid collisions).
   # With issues: populate the comments array. Without issues: leave it empty ([]) and
   # write a short "nothing actionable" body.
   cat > "/tmp/pr-review-${PR_NUMBER}-$$.json" << 'EOF'
   {
     "commit_id": "<HEAD_SHA>",
     "event": "COMMENT",
     "body": "Short review summary (1-2 sentences). When no issues were found, state that plainly.",
     "comments": [
       {
         "path": "relative/path/to/file.rs",
         "line": 42,
         "body": "Comment text — see tone guidelines below"
       }
     ]
   }
   EOF
   
   # Post the review (all comments appear as a single review). Always run this call,
   # even when "comments" is [].
   gh api repos/{owner}/{repo}/pulls/{number}/reviews --method POST --input "/tmp/pr-review-${PR_NUMBER}-$$.json"
   

   Key points:

   • Always POST the review, even with an empty comments array, so authors see the review ran.
   • Use a heredoc with 'EOF' (quoted) to prevent shell interpolation of $, backticks, etc.
   • The line field refers to the NEW file line number (right side of diff) for added/modified lines.

False Positives (skip these)

• Pre-existing issues not introduced by this PR
• Issues on lines not modified in the PR
• Things linters/compilers catch (imports, types, formatting)
• Pedantic nitpicks a senior engineer wouldn't flag
• Intentional functionality changes related to PR purpose
• General quality concerns (test coverage, docs) unless explicitly required

Notes

• Do NOT run builds/tests - CI handles that
• Use gh for all GitHub interaction
• When posting comments, post them to the specific lines of code
• Make a todo list to track progress

---

Source: a16z/jolt — distributed by TomeVault.