| name | review-reviewers |
| description | Outcome-based analysis of tend's CI behavior — checks whether tend's outputs were accepted or rejected, escalating to session logs only when outcomes look wrong. |
| argument-hint | <owner/repo> |
| metadata | {"internal":true} |
Review Reviewers
Analyze tend's CI behavior on the target repo over the window Step 1 returns. Focus on outcomes — what the bot produced publicly and whether it was accepted — rather than internal session mechanics. Create PRs or issues on tend when outcomes reveal behavioral problems.
First steps
Load /tend-ci-runner:running-in-ci first — it contains CI security rules, PR/comment formatting (line wrapping, heredoc hazards), and polling conventions. This skill opens PRs and issue comments on tend, so those rules apply.
Cost discipline: smaller, cheaper models for exploration
Session log parsing and outcome checking are token-heavy. Delegate all broad exploration to a smaller, cheaper model. Keep the main agent for judgment: evaluating findings against gates, deciding whether to act, and drafting PRs.
Pattern:
- Main agent sets up context (bot identity, repo guidance, run list)
- Main agent delegates the run survey to a subagent using a smaller, cheaper model → receives structured summary
- Main agent evaluates the summary against gates
- If needed, main agent delegates specific session logs to another subagent using a smaller, cheaper model → receives diagnosis
- Main agent drafts fix PR if warranted
Core principle: outcomes over internals
The bot's job is to produce useful outputs: reviews, triage comments, fix commits, issue responses. The cheapest way to evaluate quality is to check whether those outputs were accepted (merged, kept, acted on) or rejected (reverted, closed, corrected, disagreed with).
Session logs are expensive to download and parse. Only escalate to session-log inspection when outcome signals indicate a real problem worth diagnosing.
Core principle: repo-specific guidance is primary
Each adopter repo has its own guidance (running-tend skill or equivalent) that shapes how the bot should behave in that repo. This repo-specific guidance takes precedence over tend's default rules. The bot's job is to follow the repo-specific guidance first, falling back to tend's defaults only where the repo doesn't specify.
Non-issues: do not flag these
Some patterns look suspicious but are intentional — flagging expected behavior creates maintainer churn and costs trust. Three structural rules cover them:
-
Designed no-ops. Many events correctly end with nothing posted, at whatever layer catches them: a pre-boot gate skip (tend-mention's verify gate on the bot's own comments and reviews — though targets on older pinned releases still boot sessions for those), or a session that boots and exits silently (tend-triage on the bot's own monthly tracking-issue creation; the issue_comment.edited retrigger after a commenter refines their comment — the edit can change relevance, so the retrigger must re-evaluate; tend-notifications mark-reading a cross-repo ci_activity notification from an abandoned fork). These cost compute, not correctness — Gate 3 classifies them waste-class: record and move on; do not propose a skip-gate, label filter, pre-check, or occurrence threshold to save the boot. A loop that produces wrong outward actions (duplicate comments, spurious reviews) is different — that passes Gate 3, and a label-based skip is preferred over an authorship filter where a label can express it.
-
Designed silence. The bundled review skill authorizes posting nothing when there is nothing actionable: on a self-authored PR (GitHub rejects self-approvals, so APPROVE isn't an option), on a draft PR (COMMENT-only mode; GitHub blocks approving drafts), or when the PR closed or merged while its run was queued. GitHub reports drafts as state: OPEN, so before reading a missing review as omission — or escalating to session logs to explain it — check gh api repos/OWNER/REPO/pulls/N --jq '{state, draft}' and the PR's literal author (gh pr view <n> --json author --jq '.author.login'; owner-authored PRs are approved normally and are no bot-authored-APPROVE precedent).
-
The reviewer role is independent of authorship. tend-review re-reviewing — and re-approving — after any tend workflow pushes a fix commit is the design, not a re-approval loop; authorship-keyed guards that skip re-review drop real work and are not an accepted shape. Stacked approvals from racing runs are a concurrency artifact (cancelled runs POSTing before the SIGTERM arrived), not a review-rule problem.
Target repo
Target repo: $ARGUMENTS
Analysis targets an adopter repo whose CI runs are analyzed. Findings result in PRs/issues on the current repo (tend) to improve skills and workflows.
Use -R $ARGUMENTS for commands that access the target repo (querying runs, PRs, issues). Commands without -R default to tend.
@review-gates.md
Evidence accumulation
Evidence lives in one secret gist per target repo and month, indexed by the
monthly review-reviewers-tracking issue on Tend. Prepare it and read the
current and previous month's evidence:
uv run --script \
"${CLAUDE_PLUGIN_ROOT}/scripts/review_reviewers.py" \
prepare-evidence "$ARGUMENTS"
The command finds or creates both index and gist, announces a new gist once,
persists their ids, and prints both evidence windows.
After applying the gates, write this run's findings in the format from
@review-gates.md to $TMPDIR/findings.md. Append a ## Run $GITHUB_RUN_ID heading every run, including an all-clear window; the heading
is the audit trail future runs use. Then append it:
uv run --script \
"${CLAUDE_PLUGIN_ROOT}/scripts/review_reviewers.py" append-evidence
The command refuses a findings file that does not name this run, fetches the
latest gist content, and appends without replacing prior evidence.
Step 1: Setup
Resolve the target repo's bot login and load repo-specific guidance upfront — both are needed throughout. gh api user returns the analysis bot (e.g., tend-agent when review-reviewers runs on tend), which is typically not the target repo's bot — filtering reviews/comments by the wrong login produces false "no bot output" negatives. Read bot_name from the target repo's .config/tend.yaml:
BOT_LOGIN=$(gh api "repos/$ARGUMENTS/contents/.config/tend.yaml" --jq '.content' 2>/dev/null \
| base64 -d 2>/dev/null \
| yq '.bot_name // ""' 2>/dev/null)
if [ -z "$BOT_LOGIN" ]; then
echo "ERROR: could not resolve bot_name from $ARGUMENTS/.config/tend.yaml" >&2
exit 1
fi
echo "BOT_LOGIN=$BOT_LOGIN (target: $ARGUMENTS)"
Read the target repo's repo-specific guidance to understand what the bot was told to do:
gh api "repos/$ARGUMENTS/contents/.claude/skills/running-tend/SKILL.md" \
--jq '.content' | base64 -d
If the file doesn't exist, try the legacy overlay paths and the repo's root project instructions (.claude/skills/running-tend.md, .claude/CLAUDE.md, CLAUDE.md, AGENTS.md). Understanding the repo's guidance is essential context for evaluating outcomes — without it, you'll misjudge authorized behavior as a violation.
Then list recently completed tend CI runs on the target repo:
TARGET_REPO=$ARGUMENTS uv run --script \
"${CLAUDE_PLUGIN_ROOT}/scripts/list_recent_runs.py" review-reviewers
The script discovers tend-* workflows by default. Pass additional prefixes as arguments to include other workflows (e.g., review-reviewers when analyzing tend itself).
If empty, record the run as all-clear per "Recording below-threshold findings" above, then skip to Step 6.
If the script printed a WARNING: on stderr, the list is known-incomplete — the window was clamped, no anchor was found, or a workflow hit the fetch limit. Record a coverage gap naming the missing span instead of an all-clear, whether or not the list came back empty; the next run's floor advances past that span regardless, so an unrecorded gap is never revisited. If the script fails (non-zero exit, e.g. a transient API error), re-run it once; if it fails again, record the window as a coverage gap the same way — this run still concludes green, so the next tick anchors on it and never revisits the span.
State the window you analyzed. Its floor is the previous successful run of this workflow, or 6h back when that is older. This workflow is dispatch-only, so runs sit further apart than a cron's and the floor moves accordingly: scope every claim to it — "no problems since 08:12Z", never "no problems" — and say plainly when the run was dispatched to check on something that landed before it.
Step 2: Survey outcomes via a smaller, cheaper model
Spawn a subagent using a smaller, cheaper model to check outcomes across all runs from Step 1. The subagent does the token-heavy work of mapping runs to PRs/issues and checking acceptance signals.
Use a smaller, cheaper model for the subagent and a prompt like:
Survey bot outcomes on $ARGUMENTS for the following runs: [run IDs from Step 1].
The bot's login is $BOT_LOGIN.
For each run, determine:
- Did the bot produce visible output (review, comment, issue action, commit)?
- If yes, was the output accepted or rejected?
Sweep the window repo-wide before mapping any run, and report the row counts. The per-run mapping below walks run → branch → PR → endpoint; a break anywhere in that chain returns empty for every run at once, and uniform silence reads as a quiet hour rather than as a broken query. These calls take no run ID, so they fail independently of it:
WINDOW_START=<window start, ISO 8601>
WINDOW_END=<window end, ISO 8601>
IN_WINDOW="[.[] | select(.user.login == \"$BOT_LOGIN\")
| select((.created_at // .submitted_at) >= \"$WINDOW_START\"
and (.created_at // .submitted_at) <= \"$WINDOW_END\")] | length"
gh api "repos/$ARGUMENTS/issues/comments?since=$WINDOW_START&per_page=100" --jq "$IN_WINDOW"
gh api "repos/$ARGUMENTS/pulls/comments?since=$WINDOW_START&per_page=100" --jq "$IN_WINDOW"
CANDIDATES=$(gh -R $ARGUMENTS pr list --state all --limit 100 \
--search "updated:>$WINDOW_START" --json number --jq '.[].number')
echo "$CANDIDATES"
for pr in $CANDIDATES; do
gh api "repos/$ARGUMENTS/pulls/$pr/reviews?per_page=100" --jq "$IN_WINDOW"
done | jq -s add
Filter the comment calls on created_at at both ends. since is an updated_at floor with no ceiling, so unfiltered it also returns comments written days earlier and merely edited in the window, plus everything posted between the window end and now — the session's own runtime keeps that gap open, so on a busy repo it catches rows in most windows. Those are rows the per-run walk was right not to reach, and a check that contradicts a correct walk every run stops being believed. Leave CANDIDATES unbounded above: updated: matches the PR's own updated_at, so a range drops any PR that got bot output inside the window and was touched after it, and over-inclusion in a candidate list costs nothing. --limit 100 is load-bearing — gh pr list defaults to 30 and truncates silently.
The comment endpoints cover conversation and inline-review comments only; neither returns review submissions, and an empty-body APPROVE is tend-review's most common output. Without the fourth count an approvals-only window reports 0, 0 and satisfies the gate below with two zeros carrying no signal. Report all four numbers at the top of your summary. If the sweep found in-window rows your per-run walk did not reach, the walk is wrong — re-map from the PR numbers the sweep found rather than reporting those runs as silent.
Acceptance needs a non-bot actor — name it. For every acceptance or rejection signal, report the login that produced it: who merged, who reviewed, who replied, who pushed the follow-up. $BOT_LOGIN reviewing, replying to, or pushing to its own PR is the bot talking to itself, not acceptance. A non-bot merge is acceptance even when every commenter is the bot — the most common shape here is a maintainer merging a bot PR without ever posting. Reserve bot-only — no human signal for threads where $BOT_LOGIN is the only login across every surface. One gh pr view covers them all — merge actor, reviews, inline comments, conversation comments, commits:
gh -R $ARGUMENTS pr view <pr> --json number,state,author,mergedBy,reviews,comments,commits --jq '{
pr: .number, state, author: .author.login, merged_by: .mergedBy.login,
reviews: [.reviews[] | {login: .author.login, state}],
logins: ([.mergedBy.login, .reviews[].author.login, .comments[].author.login,
.commits[].authors[].login] | map(select(. != null and . != "")) | unique)
}'
logins is the bot-only test: ["$BOT_LOGIN"] or empty means no human touched the thread. Anything else, name that login and say which surface it came from. Every inline review comment belongs to a review record — including a standalone reply posted through the replies endpoint — so reviews covers inline commenters as well as submitted reviews, and its state gives the accept/reject direction. comments and reviews paginate in full. commits does not: gh pr view selects commits(first: 100) and never fetches a second page, oldest-first, so past 100 commits the newest drop out — exactly where a human follow-up push sits. On a PR that long, read the authors from gh api "repos/$ARGUMENTS/pulls/<pr>/commits" --paginate (itself capped at 250) before calling the thread bot-only.
A comments-only check is not enough on its own: it misses both a silent maintainer merge (mergedBy set, nobody commenting) and a human CHANGES_REQUESTED that carries no inline comments.
How to map runs to outputs:
Negative outcome signals — report any sign the bot's output was rejected, corrected, or ignored. Common shapes (use judgment for signals not listed):
- Human reviewer posted CHANGES_REQUESTED after bot approved
- PR closed without merge shortly after bot approved
- Bot posted no review despite a
tend-review run completing on an open PR — first check whether designed silence applies (draft or self-authored PR), then inspect the session logs if it does not.
- Subsequent commits reversed changes the bot approved
- Bot-closed issue was reopened
- Fix commit was reverted or CI still failing after bot pushed
- Human replied to bot with correction or complaint
- Bot comment contains corruption (literal
${, unescaped bangs, backslash-backticks, broken heredoc markers)
Corruption-scan recipe. Save bot bodies to a file, then scan with grep:
mkdir -p "$TMPDIR/bot-output" && : > "$TMPDIR/bot-output/all.txt"
for n in <pr-or-issue-numbers>; do
gh api "repos/$ARGUMENTS/issues/$n/comments?per_page=100" \
--jq ".[] | select(.user.login == \"$BOT_LOGIN\" and .created_at > \"<window-start>\") | \"=== #$n issue-comment \(.id) ===\n\(.body)\n\"" \
>> "$TMPDIR/bot-output/all.txt"
done
for n in <bot-opened-issues>; do
gh api "repos/$ARGUMENTS/issues/$n" \
--jq "select(.user.login == \"$BOT_LOGIN\" and .created_at > \"<window-start>\") | \"=== ISSUE #$n body ===\n\(.body)\n\"" \
>> "$TMPDIR/bot-output/all.txt"
done
for n in <bot-opened-prs>; do
gh api "repos/$ARGUMENTS/pulls/$n" \
--jq "select(.user.login == \"$BOT_LOGIN\" and .created_at > \"<window-start>\") | \"=== PR #$n body ===\n\(.body)\n\"" \
>> "$TMPDIR/bot-output/all.txt"
done
for n in <pr-numbers-bot-reviewed>; do
gh api "repos/$ARGUMENTS/pulls/$n/reviews" \
