auto-codex-review

name	auto-codex-review
description	Automated Codex review loop: after code changes, repeatedly call Codex for review and fix issues until clean. Generates review summary document and git commit. Trigger when code implementation is complete and ready for quality assurance, or when user says 'auto review', 'codex review loop', 'review and fix'.
user-invocable	true

Auto Codex Review Loop

Automated code review cycle: Codex reviews → Claude fixes → re-review → repeat until clean → report.

When to Use

• After completing a feature or bug fix implementation
• When user requests automated review (auto review, codex review, review and fix)
• As the final quality gate before deployment

Backend Detection

Before starting, detect which backend to use:

1. Check for Codex MCP tools: Look through your available MCP tools for any with "codex" in the server/tool name (e.g., codex_exec, codex:execute, codex_run).
2. If MCP tools found → set USE_MCP = true. Call Codex through MCP tools directly (faster, no subprocess overhead).
3. If no MCP tools → set USE_MCP = false. Fall back to CLI scripts (Python + codex exec subprocess).

This detection happens once at the start. Use the same backend for the entire session.

Process

┌─────────────────────────────────────────────┐
│  1. Collect code (git diff or source files) │
│  2. Call Codex review (structured JSON)     │
│  3. Parse issues (P0/P1/P2)                │
│  4. Any P0 or P1 issues?                   │
│     ├─ YES → Fix all issues → run tests    │
│     │        → go to step 2                │
│     └─ NO  → Generate summary report       │
│  5. Write review doc + git commit           │
│  6. Report to user                          │
└─────────────────────────────────────────────┘

Max rounds: 5 (safety limit to prevent infinite loops)

Step 1: Collect Diff

Send git diff output (not full file contents) so Codex reviews the actual changes.

Determine the diff range and get the diff:

Always inline pathspec exclusions on every diff command (do not store in a variable — bash does not re-parse quotes from variable expansion):

# Pathspec exclusions — inline on every command
EXCL="-- . ':!*.env' ':!secrets.*' ':!credentials.*' ':!*.key' ':!*.pem'"
# NOTE: $EXCL is shown for readability. In practice, inline the pathspecs
# directly in each command to avoid bash quoting issues.

# 1. Check for uncommitted changes first (most common pre-commit use case)
if ! git diff --quiet || ! git diff --cached --quiet; then
    # Has working-tree or staged changes — review those
    if git rev-parse --verify HEAD >/dev/null 2>&1; then
        git diff HEAD -- . ':!*.env' ':!secrets.*' ':!credentials.*' ':!*.key' ':!*.pem'
    else
        # Unborn branch (initial commit) — use staged diff only
        git diff --cached -- . ':!*.env' ':!secrets.*' ':!credentials.*' ':!*.key' ':!*.pem'
    fi
else
    # All changes are committed — diff against base
    CURRENT=$(git rev-parse --abbrev-ref HEAD)

    # Resolve default branch: use remote ref directly (works in worktree/CI)
    if git rev-parse --verify refs/remotes/origin/HEAD >/dev/null 2>&1; then
        DEFAULT_REF="origin/$(git symbolic-ref refs/remotes/origin/HEAD | sed 's@^refs/remotes/origin/@@')"
    elif git rev-parse --verify origin/main >/dev/null 2>&1; then
        DEFAULT_REF="origin/main"
    elif git rev-parse --verify origin/master >/dev/null 2>&1; then
        DEFAULT_REF="origin/master"
    elif git rev-parse --verify main >/dev/null 2>&1; then
        DEFAULT_REF="main"
    elif git rev-parse --verify master >/dev/null 2>&1; then
        DEFAULT_REF="master"
    else
        DEFAULT_REF=""
    fi

    if [ -n "$DEFAULT_REF" ] && [ "$CURRENT" != "${DEFAULT_REF#origin/}" ] && git merge-base "$DEFAULT_REF" HEAD >/dev/null 2>&1; then
        BASE=$(git merge-base "$DEFAULT_REF" HEAD)
        git diff "$BASE"..HEAD -- . ':!*.env' ':!secrets.*' ':!credentials.*' ':!*.key' ':!*.pem'
    elif git rev-parse --verify HEAD~1 >/dev/null 2>&1; then
        git diff HEAD~1..HEAD -- . ':!*.env' ':!secrets.*' ':!credentials.*' ':!*.key' ':!*.pem'
    else
        # Initial commit or shallow clone — use staged diff
        git diff --cached -- . ':!*.env' ':!secrets.*' ':!credentials.*' ':!*.key' ':!*.pem'
    fi
fi

Edge cases:

• Unborn branch / initial commit: git diff HEAD fails when there's no HEAD. The script above falls back to git diff --cached.
• Untracked files: git diff does not include untracked files. Check for them explicitly:

  git ls-files --others --exclude-standard
  

  If untracked source files exist, warn the user that they are excluded from the diff-based review. Optionally read their content directly (via the Read tool) as supplemental context. Do NOT stage them automatically (git add mutates the index).

SECURITY: Do not include diffs of sensitive files. The pathspec exclusions above cover common patterns, but are not exhaustive. Before sending the diff to Codex, scan it for:

• .env, *.env, secrets.yaml, credentials.*, *.key, *.pem
• Files containing API keys, tokens, passwords, or private keys (regardless of filename)
• config/secrets.*, any file listed in .gitignore as sensitive
• If the diff contains lines matching secret patterns (e.g., API_KEY=, -----BEGIN.*KEY-----, long hex/base64 tokens), remove those files from the diff before sending

Optional context supplement: For very large diffs (>15000 chars), also read 50-100 lines of surrounding context from the most critical changed files to help Codex understand the architecture.

Also build a context string describing:

• Project purpose and architecture
• Tech stack and dependencies
• What changed and why
• Any known constraints

Step 2: Call Codex Review

MCP Mode (USE_MCP = true)

Call the Codex MCP tool directly with the review prompt. No temp files or Python scripts needed.

Build the prompt string:

You are an expert code reviewer. Review the changes shown in this diff for bugs, race conditions, data integrity risks, security issues, and production readiness problems.

DIFF:
<git diff output, max 15000 chars>

CONTEXT:
<project description + what changed, max 3000 chars>

<if round > 1>
PREVIOUSLY FIXED ISSUES (do not re-report these):
<prev_issues>
</if>

Respond in EXACTLY this JSON format:
{"issues": [{"id": "P0-1", "priority": "P0", "title": "...", "description": "...", "fix": "..."}], "strengths": ["..."], "verdict": "PASS or FAIL"}

Priority levels: P0=critical bugs/security, P1=important/race conditions, P2=style/minor.
PASS = zero P0 + zero P1. FAIL = any P0 or P1.
Focus on the actual changes, not pre-existing code.

Call the Codex MCP tool with this prompt. Parse the JSON from the response using the same rules as Step 3.

CLI Mode (USE_MCP = false)

Create input JSON at a unique temp path (use mktemp or append PID/timestamp to avoid collisions):

{
  "diff": "<git diff output>",
  "context": "<project description + what changed>",
  "round_num": 1,
  "prev_issues": "",
  "timeout": 300
}

Note: The diff field triggers diff-aware truncation (by file/hunk boundaries). The legacy code field is still supported for backward compatibility but diff is preferred.

Call the review script:

python3 $HOME/.claude/skills/auto-codex-review/scripts/call_codex_review.py /tmp/auto_review_input.json

Step 3: Parse Results

The script returns JSON with this structure:

{
  "review": {
    "issues": [
      {"id": "P0-1", "priority": "P0", "title": "...", "description": "...", "fix": "..."},
      {"id": "P1-1", "priority": "P1", "title": "...", "description": "...", "fix": "..."}
    ],
    "strengths": ["strength 1", "..."],
    "verdict": "PASS or FAIL"
  },
  "raw": "raw codex output"
}

The script recomputes verdict from issues — it does NOT trust the model's self-reported verdict. PASS means zero P0 + zero P1.

If review is null (parse failure): The script returns a parse_error field. Treat this as a FAIL round. Parse the raw field manually to extract issues. Look for patterns like "P0:", "P1:", "P2:", "Issue:", "Bug:", "Problem:" etc. If you identify issues from the raw text, construct the issue list yourself.

If error is returned (CLI failure, timeout): Log the error. Retry once. If retry also fails, stop the loop and report to user.

Record this round's results in memory for the summary:

• Round number
• Issues found (count by priority)
• Issue details (titles for tracking)

Step 3.5: Evaluate & Challenge

Before fixing issues, evaluate each one. If you have strong technical evidence that an issue is invalid, challenge it through debate.

For each issue returned by Codex:

1. Evaluate technically: Is this a genuine problem given the actual codebase?

2. If AGREE: Add to fix list, proceed to Step 4 normally

3. If DISAGREE (requires concrete technical evidence): a. Write a rebuttal with: technical reasoning + code evidence (not opinion)

   MCP Mode (USE_MCP = true): Call the Codex MCP tool directly with the debate prompt:

   You are an impartial technical judge evaluating a code review disagreement.
   
   ORIGINAL ISSUE:
   Issue ID: P1-3, Priority: P1, Title: ..., Description: ..., Fix: ...
   
   CLAUDE'S REBUTTAL:
   <technical reasoning>
   
   RELEVANT CODE:
   <code snippet>
   
   Respond in EXACTLY this JSON format:
   {"issue_id": "P1-3", "round": 1, "verdict": "WITHDRAW or DOWNGRADE or INSIST", "new_priority": "P0 or P1 or P2 or null", "reasoning": "..."}
   

   Parse the JSON response and process the verdict below.

   CLI Mode (USE_MCP = false): b. Create debate input JSON:

   {
     "issue": {"id": "P1-3", "priority": "P1", "title": "...", "description": "...", "fix": "..."},
     "rebuttal": "Technical reasoning why this issue is invalid, citing specific code...",
     "code_snippet": "relevant code that proves the rebuttal",
     "round": 1,
     "timeout": 120
   }
   

   c. Save to a unique temp path and call the debate script:

   python3 $HOME/.claude/skills/auto-codex-review/scripts/call_codex_debate.py /tmp/debate_input_ISSUE_ID.json
   

   d. Process the verdict:

   • WITHDRAW → Mark issue as dismissed, record in report with both sides' reasoning
   • DOWNGRADE → Update issue priority to new_priority, re-classify for Step 4
   • INSIST → Write a stronger rebuttal with additional evidence, call debate round 2 e. After round 2 INSIST → Stop debating. Use AskUserQuestion to present:
   • The original issue (Codex's position)
   • Claude's rebuttal (your position)
   • Codex's counter-reasoning from both rounds
   • Options: "Fix it", "Dismiss it", "Downgrade to P2" The user's decision is final.

When to challenge (valid reasons):

• Type system or compiler guarantees make the issue impossible
• Framework/library contracts ensure safety (e.g., parameterized queries prevent SQL injection)
• The code path is unreachable given call site constraints
• The issue refers to code that doesn't exist (truncation artifact)

When NOT to challenge:

• Subjective preference ("I think it's fine")
• Style disagreements (just fix P2 style issues)
• You're not sure — if in doubt, fix it

Debate rounds are independent of the 5-round review limit. Each disputed issue gets max 2 debate rounds. Debates happen before any fixes in the current review round.

Step 4: Fix or Pass

If verdict is FAIL (any P0 or P1 issues):

1. Fix ALL identified issues (P0, P1, and P2)
2. MANDATORY: Run project tests to verify fixes don't break anything. If tests fail, fix the test failures before proceeding.
3. Increment round number
4. Set prev_issues to the list of issues from this round (so Codex knows what was fixed)
5. Staleness check: Compare this round's issue titles/IDs with previous rounds:
   • P2 repeated 3x → Auto-skip (likely false positive or style preference)
   • P0/P1 repeated 3x → Stop the loop and escalate to user. Present the issue details, fix attempts, and let the user decide (fix manually / dismiss / redesign)
6. Go back to Step 2

If verdict is PASS (no P0/P1, maybe P2 only):

1. Optionally fix P2 issues (recommended if simple)
2. Proceed to Step 5

If max rounds (5) reached with issues remaining:

1. Stop the loop
2. Include remaining unfixed issues in the summary
3. Flag to user that manual review may be needed

Non-progress detection: If two consecutive rounds produce the exact same number and type of issues:

• P2-only → Treat as PASS, proceed to Step 5
• Any P0/P1 remaining → Stop the loop and escalate to user for decision

Step 5: Generate Summary Document

MANDATORY: Run project tests one final time before generating the summary. All tests must pass. If they don't, fix failures first.

Write a review summary to docs/reviews/YYYY-MM-DD-<topic>.md:

# Code Review Summary: <Topic>

**Date:** YYYY-MM-DD
**Project:** <project name>
**Reviewer:** Codex (automated) + Claude (fixes)
**Result:** PASS / PARTIAL (if max rounds hit)

## Overview

<1-2 sentences about what was reviewed>

## Review Rounds

### Round 1
**Issues Found:** X (P0: a, P1: b, P2: c)

| ID | Priority | Issue | Fix Applied |
|----|----------|-------|-------------|
| P0-1 | P0 | <title> | <what was changed> |
| P1-1 | P1 | <title> | <what was changed> |

### Round 2
...

### Round N (Final)
**Issues Found:** 0
**Verdict:** PASS

## Disputed Issues

| ID | Priority | Issue | Claude's Rebuttal | Codex R1 | Codex R2 | Resolution |
|----|----------|-------|--------------------|----------|----------|------------|
| P1-3 | P1 | Missing null check | Type system guarantees non-null | INSIST | WITHDRAW | Dismissed |
| P0-2 | P0 | SQL injection risk | Using parameterized queries | INSIST | INSIST | User: Fix |

## Summary

| Metric | Value |
|--------|-------|
| Total Rounds | N |
| Total Issues Found | X |
| Total Issues Fixed | Y |
| P0 Issues | a found / a fixed |
| P1 Issues | b found / b fixed |
| P2 Issues | c found / c fixed |
| Disputed Issues | X challenged, Y withdrawn, Z user-decided |
| Files Modified | <list> |

## Strengths Noted
- <strength from final review>
- ...

Step 6: Report and Commit

1. Report to user first with a concise summary table showing rounds, issues, and final result.

2. Ask user whether to commit the changes. Use AskUserQuestion with options like "Commit now", "Let me review first", "Skip commit".

3. If user approves commit, stage and commit:

git add <modified source files> docs/reviews/<review-doc>.md

Commit message format:

review: auto Codex review - N rounds, X issues fixed

- Round 1: found A issues (P0: x, P1: y, P2: z)
- Round 2: found B issues (P0: x, P1: y, P2: z)
- ...
- Round N: PASS

Note: Version bumping is handled by the project's own workflow, not by this skill.

Important Rules

• Do not send sensitive files to Codex. Use pathspec exclusions and scan diff content for secrets before sending. Exclude env files, credentials, API keys, and private keys.
• NEVER skip running tests after fixes. Every fix round must pass existing tests. Tests must also pass before the final commit.
• Fix ALL issues in each round, not just P0. Fixing only critical issues leads to more rounds.
• Include the previous round's issues in prev_issues so Codex doesn't repeat already-fixed items.
• Treat parse failures as FAIL rounds. If Codex returns non-JSON or invalid schema, parse the raw text manually. If truly unparseable, retry once, then stop.
• Detect oscillation/staleness. If the same issue appears 3+ times, or two consecutive rounds have identical issues, stop and flag to user.
• Track files modified across all rounds for the commit and summary.
• The review doc goes in the project's docs/reviews/ directory, not a global location.

Example Session

Auto Codex Review — my-project feature-x

Round 1/5: Calling Codex...
  Found 5 issues (P0: 1, P1: 3, P2: 1)
  Fixing all issues...
  Tests: 27/27 passed

Round 2/5: Calling Codex...
  Found 2 issues (P0: 0, P1: 1, P2: 1)
  Fixing all issues...
  Tests: 27/27 passed

Round 3/5: Calling Codex...
  Found 0 issues
  Verdict: PASS

Summary:
  Rounds: 3
  Total issues fixed: 7 (P0: 1, P1: 4, P2: 2)
  Files modified: 4
  Review doc: docs/reviews/2026-03-05-feature-x.md
  Commit: v0.4.0: review: auto Codex review - 3 rounds, 7 issues fixed