validate-pr

name	validate-pr
description	Phase 3 of the PLEXI ship pipeline. Installs a PR build, generates test instructions, handles user pass/fail/modify responses, and manages the retry loop (max 3 soft rejects before escalating to hard reject). Input: PR number. Output: approved PR number or hard-reject.
risk	medium
source	local
date_added	2026-05-20

Validate PR

Phase 3 of the ship pipeline. Manages the test loop and all rejection paths.

Entry:

• /validate-pr <pr-number> — start validation for a PR
• /validate-pr <pr-number> --attempt <n> — resume at attempt N (used internally on soft-reject retry)

Outcomes:

• Pass → sets pipeline:merge + ready → invokes /merge-pr inline
• Soft reject → push a fix commit, reinstall, re-run (max 3 attempts total)
• Hard reject → close PR, rewrite issue body, remove all pipeline labels, clean up

Reads ## Ship Log in the issue body to determine current attempt count. Never trust an argument alone — always verify against the log.

Testing notification routing: If the env var PM_PANE_ID is set (injected by PM when it dispatches this skill), route the --choice "Talk to Claude" action to that pane so the user replies in the PM pane. Otherwise default to $PLEXI_PANE_ID.

Labels are live state — never the Ship Log. Pane: plexi${PLEXI_CHANNEL:+-$PLEXI_CHANNEL} pane name "#<n> · <state>" — no digit in status (PM census uses grep -oE '[0-9]+'). States: validate, needs-you, fixing, blocked. Source .agents/skills/_lib/pipeline-slots.sh; call pipeline_slots_set validate "$ISSUE_NUMBER" "$PR_NUMBER" <status> <summary> <error> at every status change.

---

Step 0 — Read Context

gh pr view <pr-number> --json title,headRefName,number,baseRefName,state

Extract:

• BRANCH = headRefName (e.g. feature/1234-something)
• PR_NUMBER = from arg
• ISSUE_NUMBER = parse from branch name — empty in stint-first mode

Stint-first mode. A feature/stint-<id>[-<id>...]-<slug> branch has no issue: .stint/ is authoritative and issues are optional. Do not read a digit run out of that branch as an issue number — 0469 is a stint id, and gh issue view 469 returns an unrelated issue. In this mode, for the whole skill:

• ISSUE_NUMBER is empty; the PR body is the Ship Log (read/append there instead of an issue body)
• attempt counting reads the PR body: gh pr view $PR_NUMBER --json body --jq '.body'
• the issue brief is stint show <id> (title + Scope) instead of gh issue view
• skip every gh issue edit / pipeline-label call — there is no issue to label
• the testing block's Issue row becomes a Stint row: | Stint | <id> — <title> |

Read current attempt count from the Ship Log (issue body, or PR body in stint-first mode). Count **Validate attempt entries; ATTEMPT_COUNT = that number (0 if no log yet).

# issue-first
ATTEMPT_COUNT=$(gh issue view $ISSUE_NUMBER --json body --jq '.body' \
  | grep -cE '^\*\*Validate attempt [0-9]+:' || true)
# stint-first
ATTEMPT_COUNT=$(gh pr view $PR_NUMBER --json body --jq '.body' \
  | grep -cE '^\*\*Validate attempt [0-9]+:' || true)

Mark issue as actively in this phase and update pane status. /open-pr already set pipeline:validate when it chained in here, so this is a no-op on the inline path — it exists so a standalone /validate-pr <PR> on an issue still sitting at pipeline:open-pr lands in the right phase. Do not add in progress: /open-pr deliberately removes it in favor of ready, and re-adding it here just thrashes the label back and forth.

gh issue edit $ISSUE_NUMBER \
  --remove-label "pipeline:open-pr" \
  --add-label "pipeline:validate" 2>/dev/null || true
plexi${PLEXI_CHANNEL:+-$PLEXI_CHANNEL} pane name "#<n> · validate"
pipeline_slots_set validate "$ISSUE_NUMBER" "$PR_NUMBER" working "" ""

---

Step 0b — Resume Guard After User Reply

Run this before handling any pass, fail, or modify reply, especially after context compaction. A reply to a [TESTING] block is part of PR validation by default; do not treat it as a fresh alpha bug report unless the user explicitly says to leave the PR flow.

Rehydrate the PR and force all subsequent reads, edits, tests, commits, pushes, and PR installs into the feature worktree:

if [ -z "${PR_NUMBER:-}" ]; then
  VALIDATION_STATE="/tmp/plexi-validate-${PLEXI_PANE_ID:-unknown}.env"
  test -f "$VALIDATION_STATE" || { echo "Missing PR_NUMBER and validation state: $VALIDATION_STATE"; exit 1; }
  . "$VALIDATION_STATE"
fi
PR_JSON=$(gh pr view $PR_NUMBER --json headRefName,baseRefName,state,isDraft)
BRANCH=$(printf '%s' "$PR_JSON" | jq -r '.headRefName')
PR_STATE=$(printf '%s' "$PR_JSON" | jq -r '.state')
test "$PR_STATE" = "OPEN" || { echo "PR #$PR_NUMBER is not open: $PR_STATE"; exit 1; }
ALPHA_ROOT=$(git worktree list --porcelain | awk '
  /^worktree / { path=$2 }
  /^branch refs\/heads\/alpha$/ { print path; exit }
')
test -n "$ALPHA_ROOT" || { echo "Could not resolve alpha worktree"; exit 1; }
WORKTREE="$ALPHA_ROOT/worktrees/$BRANCH"
test -d "$WORKTREE" || { echo "Missing PR worktree: $WORKTREE"; exit 1; }
CURRENT_BRANCH=$(git -C "$WORKTREE" branch --show-current)
test "$CURRENT_BRANCH" = "$BRANCH" || { echo "Wrong worktree branch: $CURRENT_BRANCH != $BRANCH"; exit 1; }
case "$CURRENT_BRANCH" in alpha|beta|main) echo "Refusing to validate from protected branch: $CURRENT_BRANCH"; exit 1 ;; esac
cd "$WORKTREE"
git status --short --branch

Hard stops:

• Current checkout is alpha, beta, or main.
• WORKTREE cannot be resolved from the PR head branch.
• The PR is closed or merged.
• You are about to edit files before the guard above has passed.

While handling validation feedback, just install is banned. Reinstall only with just pr-install $PR_NUMBER from WORKTREE.

---

Step 1 — Validation Mode Gate

Check changed files and classify the validation mode:

gh pr diff $PR_NUMBER --name-only
gh pr diff $PR_NUMBER

Step 1a — Test evidence gate (check this first). Read the issue Ship Log for a **Test evidence:** block written by the /testing skill during implementation:

gh issue view $ISSUE_NUMBER --json body --jq '.body' | grep -A5 "Test evidence"

Evaluate the conclusion, then apply overrides:

Conclusion	Default decision	Override
install skippable — full coverage	diff-review mode	Install anyway if apps/ files changed or Done When requires binary
binary install required — <reason>	install	—
docs-only — no test evidence required	diff-review mode	—
block absent	fall through to install-trigger list below	—

Do not re-run cargo test in validation. The suite ran during implementation; the Test Evidence block is the authoritative record. Validation owns diff review and user acceptance, not test execution. Never re-run a suite that the Ship Log already reports as green.

Default mode: install the binary. Always run just pr-install $PR_NUMBER unless the skip gate below applies. The install cost is low and gives the user the option to manually exercise the change.

Skip install when the Test Evidence conclusion says to. The /testing skill's Step 5 conclusion rules are the single source of truth for this decision — consume the conclusion verbatim, do not re-derive it here:

• install skippable — full coverage or docs-only → diff-review mode
• binary install required → install
• Block absent → fall through to the install-trigger list below
• Override regardless of conclusion: apps/ files changed, or Done When requires binary exercise

If skipping → mark Install row as skipped — <reason>, proceed to Step 3 (write and surface the testing block).

---

Step 2 — Install

Worktree gate — run this exact sequence. No shortcuts.

# 1. Resolve WORKTREE from the alpha root — never from git rev-parse (returns CWD's worktree).
ALPHA_ROOT=$(git worktree list --porcelain | awk '
  /^worktree / { path=$2 }
  /^branch refs\/heads\/alpha$/ { print path; exit }
')
test -n "$ALPHA_ROOT" || { echo "HARD STOP: could not resolve alpha worktree"; exit 1; }
WORKTREE="$ALPHA_ROOT/worktrees/$BRANCH"
test -d "$WORKTREE" || { echo "HARD STOP: feature worktree not found: $WORKTREE"; exit 1; }

# 2. Verify the worktree is on the right branch before touching anything.
ACTIVE_BRANCH=$(git -C "$WORKTREE" branch --show-current)
test "$ACTIVE_BRANCH" = "$BRANCH" || { echo "HARD STOP: worktree branch mismatch: $ACTIVE_BRANCH != $BRANCH"; exit 1; }
cd "$WORKTREE"

# 3. Run install and capture output to check for actual recompile.
INSTALL_OUT=$(just pr-install $PR_NUMBER 2>&1)
echo "$INSTALL_OUT"

# 4. Hard stop if no compile happened — binary would be stale alpha code.
if ! echo "$INSTALL_OUT" | grep -q "Compiling plexi"; then
  echo "WARNING: no 'Compiling plexi' in output — binary may be cached from wrong worktree."
  echo "Forcing recompile..."
  # Touch every changed file to invalidate cargo's cache for this worktree.
  git diff --name-only origin/alpha...HEAD | grep '\.rs$' | xargs -I{} touch "$WORKTREE/{}" 2>/dev/null || true
  INSTALL_OUT=$(just pr-install $PR_NUMBER 2>&1)
  echo "$INSTALL_OUT"
  if ! echo "$INSTALL_OUT" | grep -q "Compiling plexi"; then
    echo "HARD STOP: still no recompile after touch. Binary source is unknown. Do not proceed."
    exit 1
  fi
fi

PR_VERSION=$(grep '^version' Cargo.toml | head -1 | sed 's/.*= "//' | tr -d '"')

Wait for completion. The binary is now installed ($PR_VERSION). Move immediately to Step 3.

HARD STOP — do not launch the PR binary. Do not tail logs waiting for app output. Do not poll, watch, or sleep. The full sequence after install is: write testing block → notify → stop. The user launches and exercises the app; that is not the agent's job.

---

No AI review in this phase

AI diff review happens once, pre-push, in /implement-stint Phase 4. This skill does not run it again — a second pass over the same diff costs a full Codex run and, in babysitter mode, duplicates what the tester pane is already doing with fresh eyes.

Carry the implementation phase's verdict into the testing block as AI_FINDINGS (the Ship Log's Codex verdict: line). If that line is missing, set AI_FINDINGS="no pre-push review recorded" and say so plainly rather than running a review here to fill the gap.

Surface the testing block as soon as the install and mergeable check are done. Do not launch the PR app, browse logs, or run broad full-suite tests unless the install or build shows a real blocker.

---

Step 3 — Write and Surface Testing Block

Spec gate: Re-read the Done When checklist (issue body, or the stint's ## Scope in stint-first mode) before writing. Pass criteria map 1:1 to checklist items. No extra criteria.

Fetch the brief:

# issue-first
ISSUE_TITLE=$(gh issue view $ISSUE_NUMBER --json title --jq '.title')
ISSUE_WHAT=$(gh issue view $ISSUE_NUMBER --json body --jq '.body' \
  | sed '/^---$/,/^---$/d' \
  | grep -v '^#' \
  | grep -v '^$' \
  | head -3)

# stint-first — same variables, sourced from stint
ISSUE_TITLE=$(stint show <id> | sed -n 's/^Title: *//p')
ISSUE_WHAT=$(stint show <id> \
  | sed '/^---$/,/^---$/d' \
  | grep -v '^#' \
  | grep -v '^$' \
  | head -3)

This is the one-line context shown at the top of every testing block so the reviewer knows exactly what they're evaluating. In stint-first mode the block's Issue row becomes | Stint | <id> — <ISSUE_TITLE> |.

Surface testing block format — binary install path:

[TESTING] PR #<n> — <title>

| | |
|---|---|
| PR | #<n> · attempt <attempt+1>/3 |
| Issue | #<issue-number> — <ISSUE_TITLE> |
| Install | done — v<PR_VERSION> |
| AI review | <one-line finding or "skipped — <reason>"> |
| Test evidence | <conclusion line, or "none"> |

What this ships: <ISSUE_WHAT — one line>

Pass: <concrete observable outcome(s), comma-separated or bullet if >2>
Fail: <concrete observable symptom(s)>

Reply: "pass" | "fail: <desc>" | "modify: <change>"

Surface testing block format — diff-review path:

[TESTING] PR #<n> — <title> (diff review only)

| | |
|---|---|
| PR | #<n> |
| Issue | #<issue-number> — <ISSUE_TITLE> |
| Install | skipped — diff review only |
| AI review | <one-line finding or "No issues found"> |
| Test evidence | <conclusion line, or "none"> |

What this ships: <ISSUE_WHAT — one line>

No binary install was run; validation is limited to the diff and Codex review.
AI findings: <AI_FINDINGS verbatim, or "No issues found.">

Reply: "pass" | "fail: <desc>" | "modify: <change>"

On the diff-review path, omit the Pass: / Fail: criteria entirely. The user is reviewing the Codex findings, not exercising a binary. Only the install path asks the user to verify observable stop-check criteria.

AI_FINDINGS always shown verbatim when a pre-push review verdict exists; if none was recorded, say so in one line rather than leaving the row blank.

Your final response for this turn must be the testing block above. Do not replace it with a status recap. Do not keep working after it is surfaced.

After printing the testing block in the final response, send a best-effort notification. The notification is only an attention cue; the testing block is the source of truth.

Use a non-blocking choice notification so the user can focus the reply pane without leaving the agent process blocked on a response file:

# Persist enough state for a later user reply to survive context compaction.
ALPHA_ROOT=$(git worktree list --porcelain | awk '
  /^worktree / { path=$2 }
  /^branch refs\/heads\/alpha$/ { print path; exit }
')
VALIDATION_STATE="/tmp/plexi-validate-${PLEXI_PANE_ID:-unknown}.env"
{
  printf 'PR_NUMBER=%s\n' "$PR_NUMBER"
  printf 'ISSUE_NUMBER=%s\n' "$ISSUE_NUMBER"
  printf 'BRANCH=%s\n' "$BRANCH"
  printf 'WORKTREE=%s\n' "$ALPHA_ROOT/worktrees/$BRANCH"
} > "$VALIDATION_STATE"

# Flip pane status to needs-you so the PM surfaces this lane as awaiting the user
plexi${PLEXI_CHANNEL:+-$PLEXI_CHANNEL} pane name "#<n> · needs-you"
pipeline_slots_set validate "$ISSUE_NUMBER" "$PR_NUMBER" needs-you "Review the [TESTING] block, then reply pass/fail/modify." ""
# Route reply to PM pane if PM dispatched this skill, otherwise this pane
REPLY_PANE="${PM_PANE_ID:-$PLEXI_PANE_ID}"
PLEXI_SOCKET="$PLEXI_SOCKET" PLEXI_CHANNEL="$PLEXI_CHANNEL" \
  plexi${PLEXI_CHANNEL:+-$PLEXI_CHANNEL} notify --no-wait \
  --title "PR #<n> quality checks done (attempt $((ATTEMPT_COUNT+1))/3)" \
  --body "<title>. Review the [TESTING] block, then reply pass/fail/modify." \
  --choice "talk:Talk to Claude:pane_focus:$REPLY_PANE"

STOP. Wait for user reply.

---

Step 4 — Handle Response

Pass

Append to issue Ship Log and set pipeline state in one call:

CURRENT_BODY=$(gh issue view $ISSUE_NUMBER --json body --jq '.body')
gh issue edit $ISSUE_NUMBER \
  --body "$(printf '%s\n**Validate attempt %s:** PASS\n**Test date:** %s' "$CURRENT_BODY" "$((ATTEMPT_COUNT+1))" "$(date +%Y-%m-%d)")" \
  --add-label "pipeline:merge" \
  --add-label "ready" \
  --remove-label "pipeline:validate" \
  --remove-label "in progress"

Output:

[VALIDATED] PR #<n> — <title>
Attempt: <N>/3
Pipeline: pipeline:merge + ready set — invoking /merge-pr inline

Invoke /merge-pr <pr-number> inline in the same pane.

Modify

Valid only if pass criteria were not fully met. If criteria were met and user wants extras: "Pass criteria are met. Filing that as a separate issue." Create the issue, then treat as pass.

First run Step 0b — Resume Guard After User Reply. Do not inspect or edit production files until cd "$WORKTREE" has succeeded.

Fix the specific change on the feature branch, commit, push:

plexi${PLEXI_CHANNEL:+-$PLEXI_CHANNEL} pane name "#<n> · fixing"
pipeline_slots_set validate "$ISSUE_NUMBER" "$PR_NUMBER" fixing "" ""
git add <files>
git commit -m "fix: <description>"
git push

Run cargo build from WORKTREE to confirm the fix compiles, then reinstall using the Step 2 worktree gate (verify branch, run install, hard stop if no "Compiling plexi" in output). Never run just install during validation. Re-surface the testing block (which flips status back to needs-you). Do not expand scope.

Append to Ship Log:

**Validate attempt <N>:** MODIFY — <description of change>

Soft Reject (fail with description)

"fail" without a description: ask for it before taking any action.

Check attempt count — same marker as Step 0. ### Attempt counts implementation attempts (written by /implement-issue and /implement-stint), not validation rounds; counting those here made the 3-strikes gate fire off a different number than Step 0 computed:

ATTEMPT_COUNT=$(gh issue view $ISSUE_NUMBER --json body --jq '.body' \
  | grep -cE '^\*\*Validate attempt [0-9]+:' || true)

If ATTEMPT_COUNT < 3 (soft reject — push a fix):

Read the PR build log first:

tail -100 ~/.plexi-pr-$PR_NUMBER/plexi.log

Report what the log shows. If the log alone can't reproduce the failure and you need to drive the live binary to confirm the fix, use the drive-host skill (boot → drive → observe → teardown) — do not hand-roll a launch.

First run Step 0b — Resume Guard After User Reply. Do not inspect or edit production files until cd "$WORKTREE" has succeeded.

Apply the targeted fix to the feature branch, commit, push:

plexi${PLEXI_CHANNEL:+-$PLEXI_CHANNEL} pane name "#<n> · fixing"
pipeline_slots_set validate "$ISSUE_NUMBER" "$PR_NUMBER" fixing "" "<failure description>"
git add <files>
git commit -m "fix: <description from failure>"
git push

Run cargo build from WORKTREE to confirm the fix compiles, then reinstall using the Step 2 worktree gate (verify branch, run install, hard stop if no "Compiling plexi" in output). Never run just install during validation.

Append to Ship Log:

**Validate attempt <N>:** FAIL — <verbatim user description>
**Fix applied:** <what was pushed> (commit <hash>)

Re-surface the testing block (attempt N+1/3) in your response first, then fire the notification again. STOP.

If ATTEMPT_COUNT >= 3 (escalate to hard reject):

Send notification:

RESULT=$(plexi notify \
  --title "PR #<n> failed 3 times — hard reject?" \
  --body "<title>. 3 attempts exhausted. Close and rewrite issue?" \
  --choice "a:Talk to Claude:pane_focus:$PLEXI_PANE_ID" \
  --choice "b:Hard reject — close PR" \
  --choice "c:Keep open — I'll review")

If b or user confirms in chat → proceed to Hard Reject. If c → STOP. Leave PR open. Remove attempt limit — user owns it from here.

---

Hard Reject

1. Mark the lane blocked and read the PR log:

plexi${PLEXI_CHANNEL:+-$PLEXI_CHANNEL} pane name "#<n> · blocked"
pipeline_slots_set validate "$ISSUE_NUMBER" "$PR_NUMBER" blocked "" "hard reject after failed validation"
tail -100 ~/.plexi-pr-$PR_NUMBER/plexi.log

2. Close the PR:

gh pr close $PR_NUMBER --comment "Closing after 3 failed validate attempts. Full context in issue #$ISSUE_NUMBER body."

3. Rewrite the issue body:

Fetch the current body and all Ship Log entries. Rewrite the body to include:

• Original spec (preserved)
• ## Prior Attempts section (required format):

  ## Prior Attempts
  
  **Attempt N:** What was tried (branch, files changed, approach).
  **Why it failed:** Root cause or observable symptom (from validate log and user description).
  **What to try next:** Specific next investigation step.
  

  One block per attempt, using the Ship Log entries as source.
• Updated ## Ship Log with hard-reject entry:

  **Hard reject:** <YYYY-MM-DD> — PR #<pr-number> closed after <N> attempts.
  

gh issue edit $ISSUE_NUMBER --body "<rewritten body>"

4. Re-label (remove all pipeline labels, reset to ready):

gh issue edit $ISSUE_NUMBER \
  --remove-label "in progress" \
  --remove-label "pipeline:implement" \
  --remove-label "pipeline:open-pr" \
  --remove-label "pipeline:validate" \
  --remove-label "pipeline:merge" \
  --add-label "ready"

5. Clean up:

just channel-clean pr-$PR_NUMBER
wtp remove feature/<branch> --force
git push origin --delete feature/<branch>

6. Output:

[HARD REJECT] PR #<n> closed
Issue #<issue-number> rewritten with Prior Attempts section
Branch feature/<branch> deleted
Issue re-labeled ready for next attempt

---

Diff-Review Testing Block (default)

Surface the diff-review testing block format from Step 3. Key differences from the install path:

• No attempt count in header (no retry loop for diff-only)
• No Pass: / Fail: stop-check criteria — those are only for binary install validation
• Show AI_FINDINGS verbatim (the pre-push verdict carried from /implement-stint); the user is reviewing those findings plus the diff, not exercising a binary

Flip pane status the same as the install path before notifying:

plexi${PLEXI_CHANNEL:+-$PLEXI_CHANNEL} pane name "#<n> · needs-you"
pipeline_slots_set validate "$ISSUE_NUMBER" "$PR_NUMBER" needs-you "Review the diff-review [TESTING] block, then reply pass/fail/modify." ""

---

Rules

• Binary install is the default. Skip only per the Test Evidence conclusion (single source: /testing Step 5), with the apps/-changed and Done-When overrides.
• This skill runs no AI diff review. Pre-push Codex review is owned by /implement-stint Phase 4 — see "Who reviews what" there. Validation owns install and user acceptance, not review.
• Cosmetic = colors, spacing, font sizes, help strings, markdown, config values, UI copy — anything where human visual verification is the only meaningful check
• Do not run cargo tests during validation unless Codex review names a specific risk that needs a specific test command
• AI_FINDINGS is always shown verbatim — never summarized or filtered
• Issue brief (ISSUE_TITLE + ISSUE_WHAT) must appear at the top of every testing block
• fail without description: ask for it before taking any action
• modify is only valid if pass criteria not yet met
• A user reply after a [TESTING] block stays inside /validate-pr by default, even after context compaction
• Before any validation fix, rehydrate the PR with Step 0b and move into WORKTREE; no edits from repo root
• just install is forbidden during validation; only just pr-install $PR_NUMBER from WORKTREE
• Every just pr-install must produce a "Compiling plexi" line. If absent, binary is cached from the wrong worktree — hard stop, touch changed files, retry. Do not surface a testing block until the recompile is confirmed.
• Always resolve WORKTREE from git worktree list --porcelain + alpha branch match, never from git rev-parse --show-toplevel (returns CWD's worktree, which may be wrong after context compaction)
• Never commit or release-bump alpha, beta, or main while handling pass, fail, or modify
• Attempt count comes from the Ship Log, not arguments
• Max 3 soft rejects. Hard reject requires explicit confirmation unless user already typed "fail" 3 times.
• Never close an issue on hard reject — only the PR closes. Issue stays open.
• PR build log must be read before any soft-reject diagnosis
• Test script (test_pr<N>.py) is deleted in merge-pr Phase 5 — never committed
• Never run test_pr<N>.py yourself — it blocks on plexi notify; use direct Bash for automated checks
• All subprocess calls in test script must use full absolute path to PR binary
• Never include tail or log-reading in user-facing testing block — read the log yourself, report findings directly
• When diagnosis needs the live binary driven (not just the log), use the drive-host skill — never hand-roll a host launch
• Cosmetic issues spotted during testing go in separate issues — not blockers or modifies