| name | upstream-release-docs |
| description | Analyze an upstream project's new release, verify changes against source code, and update documentation. Covers discovery, deep-dive into PRs/issues, docs audit, source-verified implementation, and review feedback handling.
|
| argument-hint | <owner/repo> [tag] |
Upstream Release Documentation
Analyze a new release of an upstream project and update the documentation site to reflect verified changes.
Core Principle
Verify everything against source code at the release tag. Never trust release notes, PR descriptions, PR review comments, or issue descriptions at face value. Always check the actual source code.
How you reach that source depends on what the caller gave you:
-
A local clone at the tag (an automated caller usually provides one, and names its path in the invocation): read it with the Read, Grep, and Glob tools. Prefer this; it costs no API quota. See Reading the upstream clone for the constraints.
-
No clone: fetch individual files from the API, and decode the base64 response.
gh api repos/<OWNER>/<REPO>/contents/<PATH>?ref=<TAG>
Claims from any human-written source (release notes, PR bodies, review comments) may be inaccurate, outdated, or aspirational. The source code at the tag is the single source of truth.
Input
Parse the argument to extract:
<OWNER>/<REPO>: the upstream repository (e.g., stacklok/toolhive-registry-server)
<TAG> (optional): the release tag (e.g., v0.6.3). If omitted, fetch the latest release.
Output conventions
Any output that may be rendered as a GitHub comment, PR body, or Markdown file in the docs-website repo (progress narration, SUMMARY.md, GAPS.md, commit messages, PR descriptions) must fully qualify references to the upstream repo. GitHub auto-links bare #NNN relative to the repo the text lives in, so a bare #777 in a docs-website comment links to docs-website PR #777, not the upstream PR.
- Refer to upstream PRs and issues as
<OWNER>/<REPO>#NNN (e.g., stacklok/toolhive#777), never bare #NNN.
- When the release notes body contains bare
#NNN, expand them to <OWNER>/<REPO>#NNN before echoing them back.
- Full PR/issue URLs are also fine.
Execution modes
This skill runs in one of two modes. The caller signals the mode; absent an explicit unattended signal, assume interactive. Never infer unattended mode from surrounding context.
Interactive (default): a human is present. At decision points that need product context you cannot derive from source (Phase 2 step 4), ask the user. Never write the GAPS.md, SUMMARY.md, NO_CHANGES.md, or REVIEWERS.json artifacts described below in interactive mode; surface that information conversationally instead. Those files are machine-readable handoff artifacts for an automated caller, and writing them during a local run just litters the repo root.
Unattended: no interactive user, for example a CI workflow that invokes /upstream-release-docs ... in unattended mode. Never ask clarifying questions; proceed best-effort at every decision point, and route anything genuinely unresolvable into the artifacts below.
Unattended decision-point behavior
When Phase 2 step 4 would normally ask the user for a major feature's "why", instead:
- Fetch the PR body and author with
gh pr view <NUMBER> --repo <OWNER>/<REPO> --json title,body,author. The PR body usually carries the "why" the author wrote at open time: motivation, intended consumers, design decisions.
- If the PR body references linked issues ("Closes #N", "Fixes #N", "Refs #N"), fetch the likely-context-bearing ones with
gh issue view <N> --repo <OWNER>/<REPO>.
- Write the "why"/consumer narrative directly into the relevant page using what you learned, translated into reader-facing language rather than the PR's engineering shorthand. This is best-effort; reviewers refine it later.
- Defer to
GAPS.md only when the rationale demonstrably cannot be derived from available sources: the PR points to an internal design doc you cannot access, multiple plausible consumer narratives exist and choosing one would mislead readers, or a release timeline or commitment needs product-team confirmation.
Reading the upstream clone
When a caller provides a local clone of the upstream repo at the release tag, read it with the Read, Grep, and Glob tools. Prefer it over gh api contents?ref=<tag>: it is already at the tag and costs no API quota.
Do not reach for Bash to explore it. An automated caller typically clones to a scratch directory outside the session's working directory, so ls, find, and grep run through Bash are refused there, and git -C <path> is refused everywhere because it can execute untrusted hooks from the target repo. Read, Grep, and Glob have no such restriction and are the right tools regardless.
Two consequences worth internalizing, because working around them by retrying Bash variants wastes turns and never succeeds:
- Don't try to run
git log on the clone. When the caller supplies .release-meta.json, its commits array is the release range's commit list. Otherwise derive the range from the release notes and gh api.
- Don't append
2>&1 or chain with && on any Bash call. That splits the command into parts that no longer match the caller's tool allowlist, so the call is denied even when the underlying command is permitted.
Artifacts (unattended mode only, written at repo root)
These files are read by the automated caller and spliced into the PR body. The filenames and the repo-root location are a contract with the caller; do not rename or relocate them.
GAPS.md - only if you genuinely need to defer (see above). An empty GAPS.md is worse than none; do not create it if every feature's "why" was resolvable from available sources.
-
Include only content gaps a human reviewer must fill. Exclude environment or sandbox limitations (for example, "couldn't run npm build"); the PR's CI handles those. Exclude "documented for clarity, not a gap" commentary.
-
Each entry must @-mention the PR author, skipping bot authors (renovate[bot], github-actions[bot], stacklokbot).
-
Each entry must include a paste-ready "Helper prompt for local Claude" referencing the specific file(s), the PR number for context, and the narrow piece of information the human must supply or confirm.
Entry format:
### <Feature name> (PR <OWNER>/<REPO>#123 by @alice)
<One paragraph: what's missing and why it couldn't be resolved from available sources.>
**File(s):** path/to/file.mdx
**Helper prompt for local Claude:**
> <Self-contained, paste-ready prompt referencing the file(s), PR number, and the narrow piece of info needed.>
REVIEWERS.json - written whenever .release-meta.json exists at the repo root and lists contributors. This one is JSON, not markdown, because the workflow parses it with jq to decide who gets a review request.
Read .release-meta.json first (the caller writes it before invoking you):
{
"repo": "stacklok/toolhive",
"prev_tag": "v0.42.0",
"new_tag": "v0.43.0",
"owner": "jerm-dro",
"owner_source": "merged release PR stacklok/toolhive#6333",
"contributors": ["alice", "bob", "carol"],
"commits": [
{
"sha": "8343851e9f06c0d67e315eb6aa4e9371d6ef76cf",
"subject": "Push skills unsigned until keyless signing lands (#6334)",
"author": "alice"
}
]
}
commits is the commit list for the release range: use it instead of trying to run git log against the upstream clone, which is refused (see Reading the upstream clone). It is also what tells you which commits belong to which contributor for the classification below.
When commits_truncated is true, the range exceeded what the caller could fetch in one request, so both commits and contributors are partial. Treat the release notes as the authoritative list of changes for that run, and say in SUMMARY.md that the commit list was truncated so a reviewer knows the classification may have missed someone.
Classify every login in contributors as docs-facing or not, and write REVIEWERS.json at the repo root:
{
"contributors": [
{
"login": "alice",
"docs_facing": true,
"docs_facing_shas": ["8343851e9f06c0d67e315eb6aa4e9371d6ef76cf"],
"note": "Confirm the ai-plugin timeout flag section matches what you shipped."
},
{
"login": "bob",
"docs_facing": false,
"reason": "CI workflow and test-fixture changes only"
}
]
}
docs_facing: true means at least one of this person's commits in the release range changed something a reader of the docs can observe: a CLI flag or subcommand, a CRD or config field, an API route, a default, an error message, a user-visible behavior, or anything you documented or corrected in this run. When you are unsure, classify as true. A needless review request is a minor annoyance; a missing one means a wrong page ships.
docs_facing_shas (docs-facing only): every full commit SHA from commits that made the contributor docs-facing. If GitHub cannot request the contributor as a reviewer, the workflow uses all of these SHAs to find the human mergers of the relevant upstream PRs. Include only commits you verified as reader-visible; do not list an unrelated commit merely because the same contributor authored it.
docs_facing: false is for changes with no reader-visible surface: CI and build plumbing, dependency bumps, tests and fixtures, internal refactors, lint fixes, comment-only edits. Base this on the actual diff you read in Phase 2, not on the commit message. A commit titled "refactor" that changes a default value is docs-facing.
note (docs-facing only): one short sentence naming the specific thing that person should check, in their terms. Not a summary of the release, and not a restatement of their PR title. Skip the note rather than pad it.
reason (non-docs-facing only): a short phrase naming what their changes actually were. This keeps the classification auditable when the run artifact is inspected; the workflow counts these contributors without rendering their names or reasons in the PR body.
- Include the owner in the list with an honest classification. The workflow requests a review from the owner either way, so classifying the owner
false costs nothing and keeps the classification truthful.
- Bot logins are already filtered out of
contributors; if one appears anyway, omit it.
Write this file even on a NO_CHANGES.md run: "no doc-relevant changes for this release" means every contributor is non-docs-facing, which is exactly the case where suppressing review requests matters most. Classify them all false with a reason.
The consequence of skipping this file is concrete: the workflow falls back to requesting a review from every contributor, which is the noisy committee behavior this artifact exists to prevent.
NO_CHANGES.md - if the Phase 3 impact map is empty (no doc-relevant changes for this release), write this at repo root with a one-line explanation and stop. Do not hand-edit any file.
SUMMARY.md - before the final commit, write a concise list of the hand-written doc changes you made. The caller surfaces it as the PR's "Summary of changes" so reviewers see what shipped without reading the diff. Skip it only when you wrote NO_CHANGES.md or made zero hand-edits. Keep it to 3-8 bullets, each formatted as one of:
Added <what> at <path> - new pages/sections
Updated <what> in <path> - meaningful prose edits
Swept <what> across N files in <area> - repo-wide renames, apiVersion bumps, etc.
Removed <what> from <path> - deletions
Describe one logical change as one bullet (don't enumerate each file in a sweep), and exclude auto-generated reference files the caller's refresh step updates: describe only your hand-written edits.
Phase 1: Discovery
-
Fetch the release:
gh release view <TAG> --repo <OWNER>/<REPO> --json tagName,name,body,publishedAt
If no tag was provided:
gh release view --repo <OWNER>/<REPO> --json tagName,name,body,publishedAt
-
Extract PR numbers from the release notes body (look for #NNN patterns or full PR URLs). When referring to these PRs in any subsequent output, always fully qualify them as <OWNER>/<REPO>#NNN (see "Output conventions" above).
-
Categorize changes into:
- New features: entirely new capabilities
- Changed defaults: existing behavior that now works differently
- New/changed configuration: new config options, env vars, CLI flags
- Deprecations: features or options being phased out
- API changes: new/modified endpoints, request/response formats
- Bug fixes: corrections that may affect documented behavior
- Internal/infra: CI, dependencies, refactoring (usually no docs impact)
-
Log the categorized summary for transparency, then proceed to Phase 2.
Phase 2: Deep Dive
For each PR identified in Phase 1 (skip internal/infra unless user requests):
-
Fetch PR details:
gh pr view <NUMBER> --repo <OWNER>/<REPO> --json title,body,files
-
Fetch linked issues if referenced:
gh issue view <NUMBER> --repo <OWNER>/<REPO> --json title,body
-
Understand the "why": for new features, look beyond the code to understand motivation and intended usage:
- Check linked issues for user stories, acceptance criteria, and "definition of done"
- Follow references to RFCs, design docs, or PRDs linked from issues or PR descriptions
- Identify the intended user workflow: who uses this, why, and what happens after?
- Map the full lifecycle: if the feature has a publish/produce side, actively search the source code for the consume/discover side. Check CLIs, client libraries, and related repositories. If consumption tooling doesn't exist in this release, note that explicitly. This gap must be documented rather than silently omitted.
- If the "why" and consumption story aren't clear from any source, flag this gap. Documentation that only covers the API surface without explaining purpose or workflow is incomplete
-
For major new features: when a change introduces an entirely new capability (not just a config change or incremental addition), the "why" and consumer workflow often cannot be derived from source code alone. In this case, ask the user for additional context before writing documentation:
- Request user stories, PRDs, RFCs, or design documents that explain the motivation and intended usage
- Ask who the target users are and what workflow they're expected to follow
- Ask how consumers are expected to discover and use the feature (CLI, IDE extension, API, etc.)
- Do not attempt to fabricate the "why" or consumer story from code structure alone. This produces documentation that covers the "what" and "how" but misses the perspective and voice that only comes from understanding the product intent
- Incremental changes (new config options, default changes, annotation additions) can proceed without this step
- In unattended mode, do not ask. Follow the unattended decision-point behavior in Execution modes: derive the "why" from the PR body and linked issues, write it best-effort, and defer to
GAPS.md only when it is genuinely underivable.
-
Check related repositories: components often span multiple repos. For example, a server's CRD/operator may live in a different repo than the server itself. When a release changes config structures, API surfaces, or deployment models, check whether related repos (operators, CLIs, client libraries) have also released changes that affect the documentation. Ask the user which repos are related if unclear (in unattended mode, infer related repos from the release notes and proceed best-effort).
-
Read the actual source code at the release tag to verify every claim made in the PR description. Use the local clone when the caller provided one, per Core Principle; otherwise fetch the file from the API and decode the base64 response:
gh api repos/<OWNER>/<REPO>/contents/<PATH>?ref=<TAG>
-
Note discrepancies between PR descriptions and actual code. Trust the code.
A PR body describes the moment it was written, not the release. This is a distinct failure from a PR body simply being wrong: the body was accurate when opened, then later commits, review changes, or a follow-up PR in the same release moved past it. stacklok/docs-website#1037 documented a field as "not wired up" because an upstream PR note said so; by the time the release shipped it was wired up, and the note was months stale. A caveat, limitation, or "not yet supported" claim in a PR body is the single least trustworthy kind of statement you will read, because it is a claim about absence, and absence is exactly what a later commit silently fixes. Before writing any limitation into the docs, verify in source at the tag that the limitation still holds. If you cannot confirm it, leave it out rather than repeating it.
-
Deep-verify behavioral claims: these are the most common source of documentation inaccuracy. For each feature, verify not just struct definitions but actual runtime behavior:
- API routes: Check the actual route registration code (e.g.,
r.Get, r.Post, r.Delete), not just handler names. Docs often claim endpoints exist at paths where no handler is registered.
- Required fields: Check validation code (e.g.,
if field == "" checks), not just struct definitions. A field present in a struct is not necessarily required; only fields checked in validation logic are enforced.
- Default values: Check the actual defaulting code or fallback logic, not comments or struct tags. For example, "defaults to main" may actually mean "defaults to the remote's HEAD" in practice.
- Precedence rules: Read the actual
if/else chain. For example, if commit != "" { ... } else if branch != "" { ... } else if tag != "" { ... } means commit > branch > tag, not commit > tag > branch.
- Delete/cleanup behavior: Check whether the code reassigns pointers, cascades deletes, or leaves orphans. Delete behavior is frequently mis-documented.
- Query parameters: Check whether parsed parameters are actually wired to the service layer and database queries. Parameters can be parsed from the URL but silently ignored if no service option or SQL filter exists for them.
- Removed surface: a release takes things away as well as adding them, and a removal is invisible in a release-notes list of features. Diff the previous tag against the new one for deleted user-facing symbols (CRD fields, status subresources, flags, subcommands, routes, config keys) and grep the docs for every one you find.
stacklok/docs-website#1086 documented status.referencingWorkloads in the same release that removed it, because the PR that added the surrounding feature described only what it added. Treat every deletion in the diff as a docs defect until you have confirmed no page mentions it.
- Command and subcommand strings: never assemble a command from a pattern. Every command, subcommand, and flag you write into a page must be traced to its actual registration in source (
AddCommand, flag registration, route table). stacklok/docs-website#1095 documented thv ai-plugin sync and thv ai-plugin upgrade because sibling commands had them; neither existed. If you cannot point at the line that registers it, it does not go in the docs.
- Containment/authorization direction: When documenting subset/superset checks, verify which argument is the caller and which is the resource. Getting the direction wrong produces examples that show the opposite of actual behavior.
-
Identify:
- Auto-generated content: files generated from upstream (OpenAPI specs, CLI reference docs, JSON schemas). Do not manually edit these; flag them for automated update instead. However, auto-generated reference docs (e.g., API endpoints from a swagger spec) do not replace the need for conceptual explanations, guide content, or cross-references in existing pages. A new feature with auto-generated API docs still needs: (1) a conceptual explanation of what it is and why it exists, (2) mentions and cross-references in related existing pages (intro pages, feature lists, related guides), and (3) guide content if the feature has non-trivial workflows. Only skip creating a duplicate API reference page: never skip the surrounding documentation.
- Hidden, flag-gated, and experimental features: see the readiness gate in step 11 below. This is the single most common over-documentation failure in this workflow, and the release notes will not warn you about it.
-
Inventory the new public surface. As you read the source and the regenerated reference assets, list every new or changed user-facing symbol the release introduces: CRD/struct fields, enum values, CLI flags and subcommands, env vars, config keys, and API routes. Most are already enumerated in the auto-synced reference assets (CLI .md, CRD *.schema.json, Swagger YAML) and the diff, so this is mostly transcription, not discovery. This list is the checklist the completeness pass in Phase 5 verifies against. It is the difference between documentation that is accurate and documentation that is complete: a release can ship five new config fields, and a section that explains one of them correctly passes every accuracy check while silently omitting the other four.
-
Apply the readiness gate before documenting anything. A release ships code; it does not thereby declare that code ready for readers. Three separate releases leaked experimental features into published docs (skills sync/upgrade and the lockfile, Sigstore signing for skills, desktop theme customization), and each cost a reviewer round-trip to remove. The pattern is always the same: the PR is well-written and enthusiastic, the code is real and complete, and nothing in either says "not yet". Readiness lives in the code, not the narrative.
For every feature you are about to document, check the source at the tag for these signals:
- CLI:
Hidden: true on the cobra.Command, or registration behind a build tag or an if on an env var / config flag. A hidden command is not user-facing no matter how complete it is.
- Feature flags and gates: a config key, env var, or CRD field that defaults to off/false and guards the code path. Search for the feature's entry point and read what has to be true for it to execute.
- Naming:
experimental, alpha, beta, preview, internal, unstable, wip, or x- prefixes on packages, flags, config keys, API routes, or CRD fields.
- API/CRD: an unserved or unstored CRD version, a route registered only under an experimental prefix, or a field marked
// +optional and absent from any example, defaulting, or validation path.
- Upstream signals: PR labels or titles carrying
experimental/do-not-document, or a PR body that says the flag will be removed later.
When any signal fires, do not document the feature. Instead:
- Leave existing docs alone; don't add a section, a flag row, or a passing mention.
- Record it in
SUMMARY.md as a one-line deferral ("Skipped thv skill sync - hidden command at cmd/skill.go:NN") so the reviewer can see you considered and rejected it rather than missed it. This is the same treatment as a conscious deferral in the Phase 5 completeness pass, and it is what keeps the completeness check from flagging it as a gap.
- Do not add a "coming soon" or "planned for a future release" note in its place. That is a change-log statement about an unreleased feature, and it goes stale on its own.
The exception is a flag-gated change to something already documented, where the flag's existence is itself the reader-facing fact (an opt-in for new behavior a reader must deliberately enable). Document the flag and what enabling it does; don't document unflagged internals behind it.
When a signal is ambiguous (a flag that defaults on, an experimental package name for a feature the release notes headline as shipped), document it and flag the call in SUMMARY.md so a reviewer can overrule you cheaply. Ambiguity resolves toward documenting; a fired signal does not.
-
Classify the release contributors (unattended mode only, when .release-meta.json is present). You have just read every PR's diff, which makes this the only point in the run where the classification is cheap and well-informed. For each login in .release-meta.json, decide whether their commits in the release range changed anything a reader can observe, and write REVIEWERS.json per the contract in Artifacts. Do this from the diffs you read, not from commit messages.
Phase 3: Audit Existing Docs
-
Search the documentation codebase for references to affected areas:
- Config values, env var names, CLI flags
- Feature names, API paths, version numbers
- Any terminology that changed
-
Displacement audit (find what the release makes false, not just what it adds). This is the step most often skipped, and the skipped result is documentation that adds a correct new section while leaving stale claims that now contradict it. A release PR is a delta: it tells you what was added, never which existing sentences that addition falsifies. Closing that gap is your job here, not the reviewer's.
For each new or changed feature, ask: does it add an alternative, a replacement, a default change, or an additional option for a capability the docs already describe? If yes, the feature has a displaced concept. Audit the displaced concept's vocabulary, not the new feature's:
- The new feature's terms (the new flag, config key, or feature name) only find where to add content. Search them to place the new section.
- The displaced concept's terms find where existing prose is now wrong. A feature named CIMD that adds a second client-registration mechanism falsifies sentences about the old mechanism (DCR, "Dynamic Client Registration", "register automatically"). Grep for those, not for the new feature name.
Then sweep the displaced concept's surrounding prose for exclusivity and temporal language, the phrasing that silently goes stale when a feature lands:
- Exclusivity: "only", "the only", "must", "always", "automatically", "the way to", "requires".
- Temporal/roadmap: "currently implements", "is planned", "not yet", "coming soon", "future release", "for now".
- Each hit is a candidate contradiction. Read it against the verified new behavior and decide whether it must be revised.
-
Heed the additive-only smell. If your planned edit to an existing page only appends a new section and revises no existing sentence (a +N / -0 diff on a prose page), treat that as a signal, not a success. A newly added capability almost always falsifies an existing "only / default / planned" statement nearby. Reconcile the page you are editing with the section you just added to it, then check sibling pages that cover the same concept.
-
Check STYLE-GUIDE.md for conventions. It is authoritative for terminology, naming, and page structure; AGENTS.md (symlinked as CLAUDE.md) restates a subset for agents, and the style guide wins where they differ. Also read the tech-writer skill (.claude/skills/tech-writer/SKILL.md) before drafting: classify each planned page or section by Diataxis mode, read the matching mode reference, and apply .claude/skills/tech-writer/references/anti-patterns.md while writing (release-diff drafting is especially prone to changelog framing).
-
Build an impact map, a table with these columns:
| File | Current text/value | Verified replacement | Change type |
|---|
| path | what exists now | what it should be | update/new/remove/contradiction |
Use the contradiction change type for existing statements the release falsifies (stale exclusivity or roadmap claims). Every contradiction found in the displacement audit must appear as a row. A feature that adds a new section to one page typically produces several contradiction rows across sibling pages, not just an new row on the page you extended.
-
Log the impact map for transparency, then proceed to Phase 4.
Phase 4: Implementation
Apply the approved changes.
Write the end state, not the transition. Your input is a release delta, but the reader's page is not: docs state how the product works now, not how it changed. When the displacement audit flags a falsified sentence, rewrite it to state the new behavior plainly. Don't preserve the old behavior alongside it ("previously X; as of vX.Y, Y"), and don't add version-stamped change notes or upgrade-ordering warnings to guides; that's release-notes material.
One exception: a breaking change or major behavioral change (a changed default, behavior that silently differs for existing setups) may get a clearly labeled, versioned admonition (:::info[Changed in vX.Y]) when upgraders need an explanation or action they can't infer from an error message. Keep the surrounding prose standalone about current behavior; the admonition carries only the upgrade delta and action. Changes that fail loudly with an obvious cause don't qualify; surface those to the reviewer (in SUMMARY.md when unattended, conversationally when interactive) rather than writing them into a page. Large migrations get a dedicated migration guide with short pointers from affected pages, following the guides-k8s/migrate-to-v1beta1.mdx pattern. The same discipline applies to vocabulary: PR bodies are written for maintainers, and their shorthand ("consumers", "shapes", "surface area") must be translated into the concrete components, fields, and values the reader sees.
-
Update existing pages, and prefer extending them over creating new ones. Edit files using the impact map, preserving the existing writing style and conventions. When a feature extends a capability the docs already cover (a new auth mechanism alongside an existing one, a new flag on a documented command, an additional option on a documented resource), the right move is almost always to add a section to the existing page and reconcile the surrounding prose (see the displacement audit in Phase 3), not to spin up a standalone page. A standalone page for an extension fragments the concept across two locations, duplicates context, and is a common over-documentation failure mode. Reserve new pages for genuinely standalone capabilities (see the next step).
But split instead of extending when the section would overload the host page. "Prefer extending" is a default, not a mandate to cram everything onto one page; the opposite failure is a page that accretes a section every release until it is too dense to scan and breaks progressive disclosure. Graduate the content to its own page, leaving a brief summary and a cross-link behind, when any of these hold:
- The content is a distinct subject readers would look for by its own name, not a facet of the host page's subject.
- It has its own full how-to lifecycle (setup, configure, verify, troubleshoot). In this case the concept can stay as a section on the host page while the guide becomes its own page; both can be correct at once.
- The host page is already long or already covers multiple capabilities, so adding more would bury existing content.
The test is whether the new content shares the host page's subject and keeps it focused, not extend-versus-new in the abstract.
-
Create new pages for genuinely standalone new features that lack existing documentation. Default to documenting new features rather than skipping them, but first confirm the feature is standalone rather than an extension of something already documented (if it extends an existing capability, prefer step 1):
Page placement: the docs are organized by product area under docs/toolhive/. Place new content in the correct section:
- Product-specific guides go in the relevant product section (
docs/toolhive/guides-ui/, guides-cli/, guides-k8s/, guides-vmcp/, guides-registry/).
- Third-party integration guides go in
docs/toolhive/integrations/.
- Cross-cutting concepts go in
docs/toolhive/concepts/.
- Per-MCP-server usage guides go in
docs/toolhive/guides-mcp/.
- Reference material goes in
docs/toolhive/reference/.
- Check the project's CLAUDE.md "Information architecture" section for the full placement rules.
Diataxis separation: keep document types distinct, but apply this at the right granularity. The rule is that concept and how-to content stay distinguishable, not that every feature gets its own page set. For a genuinely standalone capability, create separate pages per document type rather than one combined page. For a feature that extends existing documentation, a new section of the appropriate type within an existing page is usually correct and preferable to a new page; don't fragment a concept across a new standalone page just to satisfy the separation rule. The page types below describe the content each type should contain, whether it lives on its own page or as a section:
- Concept page (explanation): What is this feature, why does it exist, when would you use it? Lead with concrete scenarios and user personas ("If you maintain a shared MCP registry and want to let teams publish reusable tool bundles..."). Explain relationships to existing features.
- Guide page (how-to): Task-oriented, organized by user goals, not by API endpoint order. Include practical examples: realistic
curl commands, sample payloads with plausible values, expected responses, and error cases. If a feature has both producer and consumer sides, document both workflows.
- Reference page: Only create if not already auto-generated. If auto-generated API reference exists, link to it instead of duplicating endpoint listings.
Consumer workflow: this is a hard requirement, not optional. For every feature that has a publish/produce side, you must answer "then what?" in the documentation. Specifically:
- How does a consumer discover what was published?
- How does a consumer install, fetch, or use it?
- What tooling exists for consumption (CLI commands, IDE extensions, API calls)?
- If consumption tooling doesn't exist yet, say so explicitly in the docs. A single sentence closing the gap is better than silence. Example: "Skill installation via agent clients is planned for a future release; for now, the registry serves as a discovery and distribution layer."
- Readers who follow the docs to completion must not hit a dead end.
Practical examples: every guide page needs at least one end-to-end example with:
- Realistic sample data (not
foo/bar placeholders)
- The exact commands or API calls to run
- Expected output or response
- What to do if something goes wrong
Naming conventions: when the feature introduces naming rules (e.g., kebab-case identifiers, camelCase config keys), call these out explicitly with examples of valid and invalid names.
Page mechanics:
- Place each page in the appropriate product section directory (see "Page placement" above)
- Update sidebar/navigation configuration in
sidebars.ts
- Update frontmatter descriptions on all new and modified pages
- Add a "Next steps" section at the end of every how-to guide and tutorial page with 1-3 forward links
- Only skip creating a page that would duplicate auto-generated reference content (e.g., don't manually list API endpoints that are already in a swagger-rendered page)
-
Add cross-references: link new content from related existing pages and vice versa.
-
Update version references: bump version numbers in install instructions, compatibility matrices, etc.
-
Do not edit auto-generated files. If they need updating, note this for the user.
-
CRD reference updates: the Kubernetes CRD reference is partially auto-generated. If the release touches CRDs, know the split:
Fully auto-generated (do not hand-edit): the exact paths depend on the project's crds: array in .github/upstream-projects.yaml. Each entry declares an out (static/ schema dir) and pages (docs/ MDX dir). For toolhive OSS these are static/api-specs/toolhive-crds/ and docs/toolhive/reference/crds/; for enterprise or gateway CRDs they will differ. Within each pair, the generated files are:
<out>/*.schema.json - extracted JSON Schema per CRD
<out>/*.example.yaml - minimal required-fields YAML example
<out>/index.json - metadata + cross-reference graph
<out>/sidebar.json - sidebar fragment consumed by sidebars.ts
<pages>/*.mdx - per-CRD pages (including the landing index.mdx)
Plus one repo-wide generated file spanning all sets: src/components/CRDReference/all-schemas.ts, the consolidated Kind -> schema barrel consumed by <CRDFields> (emitted by scripts/generate-crd-barrel.mjs). Do not hand-edit it.
These come from scripts/extract-crd-schemas.mjs + scripts/generate-crd-pages.mjs + scripts/generate-crd-barrel.mjs, all driven by scripts/upstream-release/extract-crds.mjs. Regenerating means re-running: node scripts/upstream-release/extract-crds.mjs --id <project-id>. Do not edit the MDX or schema files directly.
Hand-written overrides (scripts/lib/crd-intros.mjs): every CRD in any project's index.json publishes automatically using schema-derived defaults. Entries in this file override those defaults to polish a page. All fields are optional:
slug: URL segment and MDX filename. Default: Kind.toLowerCase().
group: 'core' or 'shared'. Default: 'shared'.
summary: one-sentence DocCard pitch. Default: first sentence of the cleaned upstream schema description.
description: SEO meta description (80-150 chars). Default: "Schema reference for <Kind>.".
intro: markdown prose at the top of the page, with inline cross-links using [Kind](./slug.mdx) form. Default: the cleaned upstream schema description.
When the release adds a new CRD:
- The release PR auto-publishes the new CRD with schema-derived defaults and flags it in a
[!NOTE] block. No blocker.
- Review the generated page. If the upstream kubebuilder description is thin or the CRD should live in the
core group or appear higher on the landing page, add an override entry for that Kind to crd-intros.mjs and re-run node scripts/upstream-release/extract-crds.mjs --id <project-id>. Overridden entries render before defaults-only entries within each group, in the order they are declared in the file.
- Commit the intros change plus the regenerated outputs. You can also land this as a follow-up PR after the release PR merges.
When the release modifies an existing CRD: the schema/example regenerate automatically. If the CRD has no override entry, the intro prose will track the upstream description automatically. If it does have an override entry, update the intro only if the CRD's role materially shifted.
Phase 5: Validation
-
Re-verify every factual claim against source code at the tag. This is the third verification pass (after Phase 2 and Phase 4). For large doc sets, spawn parallel verification agents (one per file or topic area) to check all claims concurrently. Each agent should read the doc file and verify every factual claim (struct fields, API routes, defaults, behavioral logic) against the actual source code at the release tag. Collect and resolve any discrepancies before proceeding.
This pass must extend beyond the files you changed. Stale claims live in pages the release did not touch, which is exactly why they get missed: per-changed-file verification can never surface a contradiction in a file that has no diff. For every concept introduced or changed this release, re-grep the displaced concept's vocabulary and the exclusivity/temporal phrases from Phase 3 across the entire docs set, and verify each surviving statement against the new source-of-truth behavior. If any page still asserts the old mechanism is the only, default, automatic, or planned-future option, it is a defect even though it is not in your diff. Spawn a verification agent dedicated to this displacement check, separate from the per-file agents.
-
Verify completeness, not just accuracy. The pass above confirms that what you wrote is true; this pass confirms that what the release shipped is covered. Accurate-but-partial documentation (a correct new section that explains half a feature's surface) passes every accuracy check and is the most common silent failure of this workflow, because nothing in an accuracy review flags an omission. Take the new-public-surface inventory from Phase 2 and confirm each symbol is either:
- documented in prose (a guide, concept, or reference page mentions and explains it), or
- consciously deferred, with the reason recorded (auto-generated reference only, hidden/experimental, or a deferral entry).
Any inventoried symbol that is neither documented nor deferred is a coverage gap: document it, or record why not. Do not let a symbol fall through silently. A section that documents a feature's happy path but omits its flags, enum values, or config knobs is incomplete even when every sentence in it is accurate. For large surfaces, spawn a coverage agent that takes the inventory and the changed/related doc files and returns, per symbol, "documented at file:line" or "not found."
-
Check every new page for inbound links. A page that exists but nothing links to is unreachable by navigation and breaks the journey the docs are organized around. stacklok/docs-website#1086 shipped a new AI-plugins page with a sidebar entry and zero inbound links; a reviewer had to work out where it belonged in the CLI journey after the fact.
For each page you created this run:
- Grep the docs set for its path. A sidebar entry in
sidebars.ts is necessary but is not an inbound link; it satisfies navigation, not the journey.
- Require at least one inbound prose link from a page a reader plausibly reaches first: the section's introduction, the related how-to guide, or the concept page for the capability it extends.
- Add the outbound half too. The new page needs a "Next steps" section pointing 1-3 pages further along, so it isn't a dead end.
- Ask where in the journey phase order (install, use, secure, operate, optimize) the page sits, and link it from the phase before it. Placing a page in the sidebar without answering this is what produces an orphan with a correct-looking home.
Treat a new page with no inbound prose link as a build failure you must fix, not a nit for the reviewer.
-
Build the site: run the project's build command to check for broken links, missing references, or build errors.
-
Run linting: execute the project's lint/format commands.
-
Run /docs-review: invoke the docs-review skill on all changed and new files to catch style, structure, and clarity issues. When the review returns, do not stop or present the findings to the user. Instead, immediately apply every actionable fix yourself:
- For primary issues: edit the files to resolve them.
- For secondary issues and inline suggestions: apply the fixes directly.
- For items you disagree with (e.g., they conflict with verified source code): do not apply the suggestion, but briefly log each skipped item with a source-verified reason for auditability.
- After applying fixes, re-run formatting/linting to ensure the fixes are clean.
-
Fix any remaining issues found in the build or lint steps. Re-run validation until clean.
Phase 6: Handle Review Feedback
When receiving review comments (from humans or automated reviewers):
-
Verify every review comment against source code before acting on it. Reviewers can be wrong.
-
If a comment is correct, implement the fix and verify the result.
-
If a comment is incorrect, respond with evidence from the source code. Include the actual code snippet and the file path at the tag.
-
If a comment is ambiguous, check the source code to determine the correct behavior, then respond with your findings.
Key Principles
- Triple verification: verify during deep dive (Phase 2), before finalizing (Phase 5), and when handling reviews (Phase 6)
- Document what the release falsifies, not just what it adds: a new capability usually makes prior "only / default / automatic / planned" statements wrong. Audit the displaced concept's vocabulary across the whole docs set, and treat a purely additive (
+N / -0) prose edit as a smell. The reviewer cannot catch a sin of omission in a zero-deletion diff; the skill must (Phase 3 displacement audit).
- Write the end state, not the transition: docs describe current behavior, not the change history. No "Starting in vX.Y", no "moved from A to B", no upgrade-ordering warnings in guides. The one exception is a clearly labeled, versioned admonition for a breaking or silently-behavioral change upgraders must act on; route other migration guidance to the reviewer (Phase 4).
- Transparency: log the categorized summary (after Phase 1) and impact map (after Phase 3) for auditability, but do not stop; run all phases to completion
- Respect auto-generated content: don't manually edit auto-generated files, but always create the surrounding conceptual/guide content that auto-generated reference docs don't provide
- Separate by Diataxis type, at the right granularity: keep concept and how-to content distinguishable, but prefer extending an existing page over a new one when the feature extends a documented capability. Reserve new pages for standalone capabilities, and split a section out only when it would overload its host (Phase 4).
- Verify completeness, not just accuracy: accuracy checks never catch what you left out. Inventory the release's new public surface in Phase 2 and confirm in Phase 5 that each symbol is documented or consciously deferred. An accurate section that covers only part of a feature's surface is this workflow's most common silent failure.
- Document the full lifecycle: if a feature has producer and consumer sides, document both. Always answer "then what?" Readers who follow the docs to completion must not hit a dead end. If consumption tooling isn't built yet, say so explicitly.
- Ask for context on major features: don't fabricate the "why" from code alone. Interactive mode asks the user for user stories, PRDs, or RFCs; unattended mode derives it from PR bodies and linked issues, deferring to
GAPS.md only when genuinely underivable (see Execution modes). Incremental changes proceed autonomously in either mode.
- Lead with scenarios, not abstractions: open concept pages with concrete "who is this for and why should they care" scenarios, not abstract definitions
- Flag gaps honestly: if consumption tooling, client support, or integration isn't ready yet, say so explicitly rather than omitting the topic
- Use realistic examples: guide pages need end-to-end examples with plausible data, exact commands, and expected output, not placeholder values
- Call out naming conventions: when a feature introduces naming rules (casing, allowed characters, namespacing), document them explicitly with valid/invalid examples
- Apply the readiness gate: a release shipping code is not a declaration that readers should see it. Check for
Hidden: true, default-off flags, and experimental naming in source before documenting anything; a fired signal means don't document, and record the deferral in SUMMARY.md so it reads as a decision rather than an omission (Phase 2 step 11)
- Distrust claims about absence: "not yet supported", "not wired up", and "planned" in a PR body are the least reliable statements you will read, because a later commit in the same release silently falsifies them. Verify every limitation in source at the tag before writing it into a page (Phase 2 step 7)
- A release removes surface too: deletions never appear in a feature list. Diff the tags for removed fields, flags, subcommands, and routes, and grep the docs for each one (Phase 2 step 8)
- Never assemble a command from a pattern: every command, subcommand, and flag in the docs must trace to its registration in source. Sibling commands having a subcommand is not evidence this one does (Phase 2 step 8)
- No orphan pages: a new page needs at least one inbound prose link from a page readers reach first. A
sidebars.ts entry is navigation, not a journey (Phase 5 step 3)
- Don't document hidden features: skip features marked as hidden, experimental, or internal unless explicitly asked
- Follow existing conventions: match the project's style guide, writing voice, file structure, and naming patterns
- Be project-agnostic: this workflow applies to any upstream project and any docs site. Do not assume specific frameworks, file paths, or tools.