waiting-for-codex-review

name

waiting-for-codex-review

description

Use immediately after pushing a PR to GitHub when Codex is configured to review the branch. Codex reviews asynchronously — it leaves a 👀 reaction on the PR while reviewing, drops the 👀 when the run finishes (either replacing it with 👍 if satisfied, or removing it and posting review threads if changes are needed). This skill defines the polling loop and the response workflow: wait for Codex's verdict, address any review threads it posts with the same rigor as `post-implementation-audit` (TDD + no coverage regression), commit, push, reply inline, resolve the thread, then keep waiting until 👍. Triggers on phrases like "wait for codex review", "what does codex say", "is codex done", "address the codex feedback", "check the PR reviews", or any post-push handoff where the user expects an automated reviewer's verdict. Invoke this skill even when the user does not explicitly name Codex — any GitHub PR where a 👀/👍 reaction signals an automated reviewer follows the same pattern.

Waiting for Codex Review

Codex (the OpenAI Codex automated reviewer) reviews open PRs asynchronously. The reaction contract on the PR itself:

• Run starts → Codex adds 👀 (eyes). This is the "I'm looking" signal.
• Run finishes — approved → Codex removes 👀 and adds 👍 (thumbsup). No further action required.
• Run finishes — changes requested → Codex removes 👀 and posts one or more inline review threads. No 👍 is added. The disappearance of 👀 is the signal that this run finished with findings.
• You push a fix → Codex re-adds 👀 to start the next pass. The loop resumes.

The 👀-disappears transition is load-bearing: it's how you distinguish "still reviewing" from "review done — go look at the threads". Don't keep waiting for 👍 once 👀 has dropped; switch to the thread-handling subroutine immediately.

This skill is the polling + response loop for that contract. You stay in it until 👍 lands.

When to invoke

Invoke as soon as a PR is pushed and Codex is set up for the repository. Common trigger phrases:

• "wait for the codex review"
• "what does codex say about the PR"
• "address the codex feedback"
• "check the PR reviews"
• "is codex still reviewing"

If the user pushes a PR and gives no explicit next instruction, default to invoking this skill — the right next step is to watch for Codex.

Reaction lookups: the exact data

PRs are issues for the reactions endpoint. The bot's login depends on the install — on installs that use the ChatGPT Codex connector it's chatgpt-codex-connector; on the standalone Codex Cloud install it may be codex[bot] or similar. The first poll of this loop discovers the bot login rather than hard-coding it: any review-comment author whose handle contains codex is treated as Codex.

# Reactions on the PR itself
gh api repos/<owner>/<repo>/issues/<pr>/reactions
# → array of { user.login, content } where content ∈ {'+1','-1','laugh','confused','heart','hooray','rocket','eyes'}
# +1 == 👍, eyes == 👀

# Reviews and review threads
gh pr view <pr> --json reviews
gh api graphql -f query='
  query($owner:String!,$repo:String!,$n:Int!){
    repository(owner:$owner,name:$repo){
      pullRequest(number:$n){
        reviewThreads(first:100){
          nodes{
            id            # GraphQL node id — use this to RESOLVE
            isResolved
            comments(first:100){
              nodes{ databaseId author{login} path line body }
              # databaseId is the REST id — use this to REPLY
            }
          }
        }
      }
    }
  }' -F owner=<owner> -F repo=<repo> -F n=<pr>

Keep two ids per thread straight: thread.id (GraphQL node id, opaque string starting with PRRT_...) is needed to resolve the thread; comment.databaseId (integer) is needed to reply in-line. They are not interchangeable.

The loop

digraph codex_loop {
    poll [label="Poll reactions + threads", shape=diamond];
    thumbs [label="Codex 👍 on PR?", shape=diamond];
    eyes [label="Codex 👀 on PR?", shape=diamond];
    threads [label="Unresolved Codex threads?", shape=diamond];
    address [label="Address each thread (TDD)", shape=box];
    push [label="Commit + push fix", shape=box];
    reply [label="Reply inline + resolve thread", shape=box];
    trigger [label="Comment '@codex review'", shape=box];
    wait_review [label="Sleep — Codex still reviewing", shape=box];
    wait_start [label="Sleep — Codex hasn't started", shape=box];
    done [label="Done — hand back to user", shape=doublecircle];

    poll -> thumbs;
    thumbs -> done [label="yes"];
    thumbs -> eyes [label="no"];
    eyes -> wait_review [label="yes (still reviewing)"];
    eyes -> threads [label="no (👀 dropped → run finished)"];
    threads -> address [label="yes"];
    threads -> wait_start [label="no (pre-review, no run yet)"];
    wait_review -> poll;
    wait_start -> poll;
    address -> push -> reply -> trigger -> poll;
}

Three terminal poll states matter:

State	What it means	What to do
👍 present (👀 absent)	Codex approved	Done — hand back to user
👀 present	Codex is actively reviewing right now	Sleep, re-poll
👀 absent AND 👍 absent	Either Codex finished with findings, or it never started	Check threads — if any unresolved Codex thread exists, address them; otherwise treat as "not started yet" and sleep

Step-by-step

1. Identify the PR

If the user names the PR, use that number. Otherwise, fall back to the head branch:

gh pr view --json number,url,headRefName

Confirm the PR is open and was pushed in this session before entering the loop — don't enter the loop on a stale branch.

2. Kick off the initial review with @codex review

Codex does not always start a review automatically when a PR is opened. Some installs only trigger on explicit mention, others trigger automatically most of the time but not always. Either way, posting @codex review as a top-level comment right after the first push is the deterministic way to start the first review — and it's an idempotent no-op when Codex would have triggered on its own (Codex just sees the redundant mention and starts).

gh pr comment <pr> --body "@codex review"

Post this once immediately after entering the loop on a freshly-pushed PR, before the first snapshot/poll. This is the same trigger used in step 7 for subsequent passes — the difference is just timing (entry vs. after a fix push). If you entered the loop on a PR that already has a 👀 or 👍 reaction from Codex (e.g., the user invoked the skill mid-stream), skip this step — the review is already in flight.

3. Snapshot the current state

Before polling, capture the state so the next poll can detect new signals (not just any signals):

seen_reactions := { (user, content) ... } from /issues/<pr>/reactions
seen_threads   := { thread.id ... } from the GraphQL query

4. Poll, decide, act

Read the reaction set first, threads second — the reaction is the run-state signal, the threads are the run output.

👀 on PR	👍 on PR	Unresolved Codex threads	Decision
no	yes	(don't care)	Loop is done. Announce.
yes	no	(don't care)	Codex is reviewing right now. Sleep, then re-poll.
no	no	yes	Codex finished this run with findings — the 👀 dropped, no 👍 was added. Drop into the "address comments" subroutine.
no	no	no	Either Codex hasn't started yet (first poll after push), or every thread is already resolved and a fresh review is pending. Sleep, then re-poll.

Decision-making notes:

• The 👀 → ∅ transition is the "run finished with findings" signal. Don't keep waiting for 👍 — switch to thread handling as soon as 👀 drops and threads exist.
• Codex sometimes posts a top-level review body ("LGTM" / a short summary) without 👍. Treat that as advisory — only the 👍 reaction is the terminal approval signal.
• A new push does NOT reliably retrigger Codex. The auto-retrigger fires sometimes but not always; do not rely on it. After every fix push + thread resolution, post @codex review as a fresh top-level comment to deterministically start the next pass (see step 7 of the response workflow). If 👀 does not reappear within ~3 minutes after the comment, double-check that the push actually landed on the PR branch (gh pr view <pr> --json commits).
• Multiple Codex passes are normal. The loop must tolerate Codex finding new issues introduced by your fix. Each pass is a fresh 👀 → (👍 | ∅ + threads) cycle, kicked off by a fresh @codex review comment.

5. Address review comments (the hard part)

Each unresolved Codex thread is a finding that must land a fix in this PR — no follow-up issues, no deferral. The bar is the same as the post-implementation-audit skill (read it if you haven't):

1. Read the comment twice before you start. Then read the surrounding 30 lines of the file. Codex's comments are usually right but sometimes context-thin — your job is to understand whether the suggested change is the best outcome or whether the underlying concern can be fixed differently. If you disagree, see "Disagreeing with Codex" below.
2. Write the test first. For every code change you make, add or update a named test that fails on the current code and passes after the fix. Skipping the test is what lets the regression come back silently on the next iteration. This applies even to "obvious" fixes — obvious code regresses too.
3. Coverage must not regress. Capture the current per-line / per-region / per-branch numbers on the touched module before changing anything. After the fix, re-run the same command. If any number drops, find the missing test and add it before committing.
4. Workspace conventions still apply. Lint flags, derive lists, no_std constraints, doc comments on public items, spec-as-contract — all unchanged. Codex's suggestion is a finding, not a license to skip the project rules.
5. Commit with a thread-linked message. The commit message names the issue Codex raised, not just the file you touched. Example: fix(group): clamp role on legacy adapters (Codex review #648).

6. Push, reply inline, resolve

After pushing:

Reply on each thread you addressed. Use the first comment's databaseId as the parent:

gh api -X POST repos/<owner>/<repo>/pulls/<pr>/comments \
  -F in_reply_to=<parent_databaseId> \
  -F body=$'Fixed in <commit-sha>: <one-line summary of the fix>'

Resolve the thread via GraphQL (uses the node id, not the database id):

gh api graphql -f query='
  mutation($threadId:ID!){
    resolveReviewThread(input:{threadId:$threadId}){ thread { isResolved } }
  }' -F threadId=<thread.id>

Only resolve threads where you actually landed a fix or replied with a justification the user has approved. Resolving without a real fix breaks the next reviewer's trust in the resolved-status signal.

7. Re-trigger Codex with @codex review

After every addressed thread has been replied to and resolved, post a fresh top-level comment on the PR with the literal body @codex review. This is required because Codex does not reliably re-trigger from a new push alone — sometimes the post-push hook fires, sometimes it doesn't, and waiting silently is indistinguishable from "Codex stalled". The explicit mention is the deterministic way to start the next pass.

gh pr comment <pr> --body "@codex review"

Only post this comment after the fix push and the thread replies/resolves have all landed — otherwise Codex starts the next pass against stale state and burns a full review cycle on outdated code.

If there's nothing to address but the loop is still waiting (e.g., 👀 was never added after a push), posting @codex review is also the right escalation before pinging the user.

8. Loop

Return to step 3 (snapshot) with refreshed seen_* sets. Codex's next pass picks up the comment within ~30–90 s, re-adds 👀, and either approves (👍) or surfaces new findings. Step 2 (initial @codex review trigger) is not repeated — step 7 already posted the trigger for the next pass.

Polling cadence

Codex review runs typically complete in 1–5 minutes for a small PR, 10–20 for a large one. Pick the cadence by what you're waiting for:

• First poll after push: 60 seconds (Codex usually adds 👀 within 30–90 s).
• Routine poll while 👀 is up: 120 seconds (stays inside Anthropic's 5-min prompt cache window).
• Routine poll while neither 👀 nor 👍 is present and no threads exist: 120 seconds — Codex hasn't started yet (or finished a previous pass and a new one is pending).
• 👀 dropped + threads exist: don't poll — switch immediately to the thread-handling subroutine.
• No reaction at all after 5 minutes from push: ping the user — Codex may not be configured for this repo, or the install is stalled. Also re-confirm the push actually landed.
• 👀 has been up >15 minutes with no thread activity: ping the user — Codex may have stalled mid-run.

For dynamic-paced /loop runs, use ScheduleWakeup with delaySeconds=120 (cache-warm) until 👍 lands. Don't poll faster than 60 seconds — Codex doesn't react that quickly anyway, and the prompt cache cost dominates.

Prefer Bash run_in_background over Monitor for this loop. The verdict we wait for is a single terminal state (👍 approval, or 👀-dropped with new findings) — exactly the "one notification when the condition is true" shape Bash run_in_background is built for. Each iteration of the loop has exactly one terminal condition to watch, so one completion notification per loop pass is what we want.

Monitor is designed for ongoing event streams (one notification per state-change line) — useful when you want to know about every transition in real time, but notification delivery is best-effort and intermediate events can drop. Real-world example from this skill's own bootstrap session: the 👀 → 👍 transition fired correctly inside the monitor and was written to the script's stdout file, but the follow-up [change] and [approval] notifications never reached the conversation. The model sat waiting for an event that had already fired.

Use the script below with run_in_background: true (not Monitor). It loops until either terminal state, then exits with the verdict written to its last stdout line so a single Read on the task's output file at notification time recovers the verdict.

prev=""
while true; do
  reactions=$(gh api repos/<owner>/<repo>/issues/<pr>/reactions \
    --jq '.[] | "\(.user.login):\(.content)"' 2>/dev/null | sort)
  threads=$(gh api graphql -f query='query($o:String!,$r:String!,$n:Int!){
    repository(owner:$o,name:$r){pullRequest(number:$n){
      reviewThreads(first:100){nodes{id isResolved comments(first:1){nodes{author{login}}}}}}}}' \
    -F o=<owner> -F r=<repo> -F n=<pr> \
    --jq '.data.repository.pullRequest.reviewThreads.nodes
          | map(select(.isResolved==false
                       and (.comments.nodes[0].author.login | test("codex"; "i"))))
          | length' 2>/dev/null)

  codex_eyes=$(echo "$reactions" | grep -E "codex.*:eyes$" || true)
  codex_thumb=$(echo "$reactions" | grep -E "codex.*:\+1$" || true)

  cur="reactions=[$reactions] threads=$threads"
  [ "$cur" != "$prev" ] && echo "[change] $cur"
  prev=$cur

  if [ -n "$codex_thumb" ]; then
    echo "VERDICT=approval"
    exit 0
  fi
  if [ -z "$codex_eyes" ] && [ "$threads" -gt 0 ]; then
     
     0
  

   120

When this script completes via Bash run_in_background, the single completion notification fires. Recover the verdict with one Read on the task's output file — the last line is VERDICT=approval or VERDICT=needs_action threads=N. If the file's last line is something else (script crashed or hit the bash timeout), apply the defensive re-check below.

Defensive re-check: don't trust silence as "still reviewing"

Whether you use Bash run_in_background or Monitor, the delivery of notifications back to the conversation is best-effort. If the runtime drops the completion or change event, the loop sits waiting indefinitely for a signal that already fired. The script's stdout file is authoritative; the notification stream is convenience.

The mitigation is cheap and load-bearing: after any unusually long quiet period — more than ~10 minutes of total wait once you've seen 👀, or more than ~5 minutes since the most recent event — do a direct verification before continuing to wait or escalating to the user. Verification is two steps, in this order:

1. Read the background task's stdout file (the output path returned when the task started). If it contains a terminal line (the script's exit line, or an [approval] / [needs_action] event from a Monitor script) you missed the notification — act on it now.
2. Re-fetch PR state directly:

   gh api repos/<owner>/<repo>/issues/<pr>/reactions \
     --jq '.[] | "\(.user.login):\(.content)"'
   

   If the output shows :+1 (👍), or shows 👀 absent with unresolved Codex threads, the run completed and you missed the signal. Re-run the threads query and act on the verdict.

The general principle: the script's stdout file is the source of truth for detection; the notification stream is best-effort delivery. Treat the two as separate channels — the file is durable, the channel is lossy.

Codex is usually right but not always. When you read a thread and your read of the spec or code says Codex is wrong (or redundant, or stylistic-without-justification):

1. Do not unilaterally dismiss it. Stop and surface to the user with: what Codex says, what the spec / code says, your reasoning, and a recommendation (address as-is / push back / close-as-wontfix).
2. The user picks. Then follow through — if "push back", reply inline with the technical reasoning; if "close-as-wontfix", reply inline with the rationale before resolving; if "address", fall through to step 5 of the loop.
3. Never reply inline arguing the point without checking with the user first. Codex's threads are read by humans too — a wrong reply costs more than a one-minute clarification.

Anti-patterns

• Resolving threads without a real fix or explicit user disagreement. Marking resolved is load-bearing — the next reviewer trusts it. An unaddressed finding hidden by a resolution is invisible until production.
• Skipping the test because "the fix is obvious." Obvious fixes regress just as often as subtle ones. The test exists to prove the fix held over time, not to prove the fix was needed today.
• Replying with "fixed" only. Quote the commit sha and a one-line summary so the thread is self-contained when read later.
• Polling faster than 60 seconds. Burns prompt cache for no benefit. Codex doesn't react that fast.
• Continuing past a 👍 in case there are still threads. 👍 means Codex is satisfied with the current state — pending threads are stale. Re-verify with the user before bulk-resolving.
• Arguing with Codex inline without escalating. A wrong reply is more expensive than a one-minute user clarification.
• Hard-coding the bot login. Different installs use different handles (chatgpt-codex-connector, codex[bot], etc.). Match by case-insensitive substring instead.
• Treating background-task notifications as a complete event log. Whether you use Bash run_in_background or Monitor, the conversation-side notification stream is best-effort delivery — the script's stdout file is authoritative. After any quiet period of >5 minutes since the last event (or >10 minutes total since 👀 was seen), Read the task's output path and re-fetch reactions directly with gh api .../reactions before assuming "still reviewing." See "Defensive re-check" under Polling cadence.
• Using Monitor when you only need a terminal verdict. Monitor is designed for ongoing event streams (every state change becomes a notification). When the loop is waiting for a single terminal verdict — 👍 or new findings — Bash run_in_background is the right tool: one reliable completion notification, single Read of the output file to recover the verdict. Reach for Monitor only when you need real-time visibility into every intermediate state change.

What this skill is not

• A code-review skill. Actual fixes come from project-specific implementation skills (post-implementation-audit, test-driven-development, systematic-debugging). This skill is the wrapper that decides when those skills fire and how the result is reported back to GitHub.
• A merge skill. 👍 doesn't auto-merge — the user owns the merge decision. Stop the loop on 👍, hand back, let them merge.
• A way to delegate judgment back to Codex. If you disagree with a finding, surface to the user. Don't argue with the bot.