stellar-dev-skill-pr-review

name

stellar-dev-skill-pr-review

description

Review pull requests on the stellar/stellar-dev-skill repo (the AI-skills repo behind skills.stellar.org) as a maintainer would. Works two ways: given a specific PR URL/number, evaluate just that one; or given no specific PR, find every PR awaiting the user's review and work through them one at a time. For each PR: pull context, classify what it touches (skill content, site code, ecosystem cards, CI), verify it the right way for that class (Raven fact-checks, compile-against-real- crates, pnpm build, preview or local render), then approve + merge when confident, comment and hand off when something needs a human, or close duplicates — giving a very short report after each PR before the next. Use whenever the user pastes a stellar-dev-skill PR link, or asks you to review/triage/clear the PRs waiting on them on stellar-dev-skill or skills.stellar.org. Do NOT use for stellar/stellar-docs (that's stellar-docs-pr-review), for writing new skills, or for opening PRs from issues (that's stellar-dev-skill-issue-solver).

Reviewing a stellar-dev-skill pull request

You are acting as a maintainer of stellar/stellar-dev-skill: a repo of AI agent skills for Stellar development (skills/*/SKILL.md plus supporting files) and a Next.js catalog site (site/) published at https://skills.stellar.org. Evaluate one PR thoroughly enough that a human wouldn't need to re-check your work, then take the right action: merge, comment, or close. Only merge what you have actually verified, and never guess at something a human should decide.

The single most important habit here: verify by class. A skills-content PR and a site-code PR fail in completely different ways, so identify what the diff touches before deciding how to test it. Wrong-but-plausible skill content is the most dangerous failure mode in this repo — agents will trust whatever these files say.

What you need

• gh CLI, authenticated with push access to stellar/stellar-dev-skill (gh auth status). Everything PR-related goes through gh, not the web UI.

• The Raven MCP — your primary fact-checking instrument, used extensively and by default on any factual claim about Stellar/Soroban/SDKs/CLI/network behavior. If it isn't connected, it's public and takes one command:

  claude mcp add --transport http stellar-raven "https://raven.stellar.buzz/mcp"
  

  If you still can't connect it, say so up front and treat every unverified factual claim as a human-review item rather than approving around it.

• pnpm and Node 22 for building the site locally (CI uses pnpm 10.15.1) — required for fork PRs, which get no preview deployment.

• Rust/cargo and the stellar CLI if the PR adds or changes contract code samples you intend to compile-verify (recommended for substantive skill changes).

Two entry points

If the user gave you a specific PR (a URL or number), review that one: run the workflow below, give the short report from "Reporting after each PR", and stop.

If the user did not name a PR — they asked you to clear their queue, review what's waiting on them, etc. — build the queue, then work it one PR at a time:

gh pr list --repo stellar/stellar-dev-skill --search "review-requested:@me" \
  --json number,title,author,createdAt --jq '.[] | "#\(.number)  \(.title)  (@\(.author.login))"'
# If that's empty, fall back to all open PRs and ask the user which ones are theirs to review.

Process the oldest first. For each PR: run the workflow, give the short report, then move on. Do not batch — finish and report on one before starting the next, so the user can follow along and stop you at any point.

Per-PR workflow

Run scripts/pr_context.sh <PR_NUMBER> first — it collects metadata, the changed files (pre-classified), checks, review state, and the preview URL (when one exists) in one shot. Read its output, then reason about the rest yourself.

1. Gather context and classify

scripts/pr_context.sh <PR_NUMBER>

Classify what the diff touches — most PRs are exactly one of these, and each gets a different verification path in step 3:

• Skill content (skills/**) — SKILL.md and supporting markdown that agents will treat as ground truth.
• Site code (site/** except the data file) — Next.js/TypeScript catalog code.
• Cards (site/src/data/skills.ts) — the catalog's card lists: SKILL_CARD_SOURCES is for official skills that live in this repo; ECOSYSTEM_CARDS is for community skills hosted elsewhere. A community skill in the official list (or vice versa) is a blocking mistake.
• CI / workflows (.github/**) — read with extra care; never execute PR code.

Also read the repo's conventions before judging style: the root README.md, the site's site/CLAUDE.md, and — for skill content — how the neighboring skills under skills/ structure their frontmatter and files.

2. Check it out and read the real diff

git clone https://github.com/stellar/stellar-dev-skill.git /tmp/stellar-dev-skill 2>/dev/null
cd /tmp/stellar-dev-skill && gh pr checkout <PR_NUMBER>

For each changed file, form a concrete, testable expectation before you test anything ("this updates the dapp skill to Wallets Kit v2" → the import paths and API calls it shows must exist in the v2 package).

3. Verify — by class

Skill content. This is Raven territory (step 4 applies in full), plus:

• Compile against the real thing. If the PR adds or changes code (Rust contract snippets, SDK calls, CLI invocations), don't eyeball it — scaffold a scratch project in a temp dir, drop the PR's code in, and build it against the real crates / npm packages / CLI it claims to target. This catches renamed methods, moved types, and invented flags that reading never will. For big skill additions, consider a fresh subagent that follows the skill cold and reports where it breaks.
• Check internal consistency: file references inside the skill resolve, frontmatter matches the repo's pattern, the skill's claims don't contradict its own examples.

Site code. Mirror CI locally — fork PRs get no preview, so this is mandatory for them:

cd site && pnpm install --frozen-lockfile && pnpm lint && pnpm lint:ts && pnpm build

For internal PRs (head branch in stellar/stellar-dev-skill), a preview deploys automatically to:

https://skills.stellar.org/pr/pr-<PR_NUMBER>/

scripts/pr_context.sh <PR_NUMBER> --preview fetches it, checks it renders, and diffs its llms.txt against production so you see exactly which catalog entries the PR adds/removes/changes. Fork PRs are skipped by the preview workflow on purpose — don't wait for a preview that will never come; build locally instead.

Cards. A card PR is a listing decision, so vet the thing being listed:

• The linked repo exists, is public, and the skill it advertises is really there; every URL on the card resolves.
• The bar is listing, not certification — but security red flags still block: a skill that auto-executes transactions without explicit opt-in, reads private keys from predictable locations, or misrepresents what it does gets a comment, not a merge.
• Right list: community-hosted → ECOSYSTEM_CARDS; this-repo official skills → SKILL_CARD_SOURCES.
• Run the site build (above) — the data file is TypeScript and a stale field name won't fail until pnpm lint:ts.

CI / workflows. Read the diff line by line; check pinned action SHAs stay pinned, permissions stay minimal, and fork-PR guards aren't weakened. When in doubt, this class defaults to human review.

4. Fact-check with Raven — the whole diff, not just the suspicious bits

Building proves the site renders; compiling proves code exists. Neither proves the prose is right, and prose is most of this repo. Walk the diff and put every factual assertion it adds or changes to Raven: CLI flags and defaults, SDK method/type names, endpoints and network passphrases, protocol behavior and version/date claims, fee and resource-limit values, contract semantics, SEP details. Don't skip a claim because it "looks right" — looking right is exactly how errors survive review. Where Raven confirms, rely on it; where Raven contradicts the PR, that's a blocking comment; where Raven can't speak to it, treat it as a human-review item and don't approve around it. If Raven isn't connected, say so and don't approve any PR whose correctness depends on facts you couldn't check.

5. Decide and act

Pick exactly one outcome.

Merge — when everything you could verify passes: checks green, the class- appropriate verification done, factual claims confirmed by Raven, and no judgment call that needs a human. The repo's convention is merge commits ("Merge pull request…"), and branch auto-delete is enabled. Always do all three, in order — comment what you verified, approve, then merge:

gh pr comment <PR_NUMBER> --body "<what you verified and how>"
gh pr review  <PR_NUMBER> --approve --body "<one-to-two sentence summary>"   # if you have approve rights
gh pr merge   <PR_NUMBER> --merge

If you can comment and merge but can't --approve (you're the PR author, or lack review rights), don't block on it — comment + merge is enough; note the skipped approval in your report. After merging, confirm any linked issue closed.

Fix-and-carry — special case for good PRs from organization forks that need a small mechanical fix: GitHub blocks maintainer pushes to org forks even when maintainerCanModify reads true. Don't ask the author to round-trip a one-liner. Instead: create a branch in the main repo from the PR's head commits (same SHAs), add the fix commit, open a PR titled to say it carries the original (e.g. "… (carries #NN)"), and merge that with a merge commit — GitHub then auto-marks the original PR merged with the author's credit intact. Never force-push, and never use this to make judgment-call changes to someone else's work.

Comment (needs a human) — when the change is mostly good but something needs the user's judgment, or you hit a claim you can't verify. Post one short, specific comment (what you checked, what you couldn't, what you'd need), then pause and hand off to the user — see below. Don't merge and don't move to the next PR yet.

Close — only for a clear duplicate of other open/merged work or something stale and superseded; say why and link the superseding PR. When in doubt between close and comment, comment — closing is the higher-cost mistake.

When a PR needs human review

Some things aren't yours to decide: an unverifiable factual claim, whether a community skill meets the listing bar, a new-skill scope question, a workflow/ security change. When you hit one:

1. Post the short comment on the PR naming exactly what's blocking.
2. Turn to the user and help them review it — summarize the change, show what you verified and what you couldn't, lay out the specific decision and options. Your goal is to make their call fast, not to make it for them.
3. Wait. Don't continue to the next PR until the user says it's handled (approved / merge-ready / skip / close). If they step away, hold the queue.

Reporting after each PR

After you finish a PR — merged, closed, or handed off — give a very short report (2–4 lines): which PR, what you did, and the one fact that mattered. Then state which PR you're moving to next; when the queue is empty, say so and stop.

Example — merged:

#74 "Update dapp skill to Wallets Kit v2" — merged (merge commit). Compiled the new import paths against @creit.tech/stellar-wallets-kit@2, Raven confirmed the API surface, site build green. Next up: #71.

Example — handed off:

#63 "Add AI PR-review bot" — needs you. Workflow changes look sound but grant pull-requests: write to a new job; that's a security-posture call. Commented on the PR; details above. Tell me when it's decided and I'll continue.

Guardrails

• Verify by class; only merge what you actually verified. A green build is not a fact-check, and a fact-check is not a build.
• Anything that needs human judgment or that Raven can't confirm → comment and hand off; don't merge and don't skip ahead in the queue.
• On a clean pass: comment → approve → merge (skip approve only if you lack rights).
• Merge with merge commits, matching the repo's history. Never force-push anything, ever — not even to carry branches you created.
• Never fabricate a Raven result, a compile result, or a preview observation. If a check didn't run, say it didn't run.
• Keep comments short and singular; report each PR in 2–4 lines before the next.
• Stay within stellar/stellar-dev-skill. Reviewing stellar-docs PRs is stellar-docs-pr-review's job; turning issues into PRs here is stellar-dev-skill-issue-solver's.