review-existing-content

# Review Existing Content You are reviewing existing documentation pages — not a PR diff. The selection script has already chosen today's articles; your job is to run the docs-review machinery over each whole file and apply only the fixes you can defend with an authoritative source. Everything judgment-level goes in the PR description for a human, not in the diff. You run **unprivileged**: the review job holds no push token, so you edit the working tree only — never `git commit`, `git push`, or `gh pr create`. The workflow's publish job validates your changes against a deterministic gate (`scripts/content-review/publish-gate.py`: verdict shape, diff scope, `no_retire`), derives the branch name from the queue, and opens the PR (ready when the re-lint passes, draft only on lint failure) with the body you edited. Changes outside the gate's scope are rejected wholesale, so keep every edit inside the bounds each step names. ## Input Read `.content-review-queue.json` from the repo root (written by `scripts/content-review/select-articles.py`). Shape: ```json { "generated": "2026-06-12T14:00:00+00:00", "count": 3, "mode": "fix", "halted": null, "traffic": { "available": true, "period": "2026-05", "pages_matched": 731 }, "reader_signals": { "available": true, "gsc": { "available": true, "period": {"start": "2026-03-14", "end": "2026-06-11"}, "pages_matched": 612, "median_ctr": 0.031, "max_impressions": 88012 }, "feedback": { "available": true, "pages_matched": 214 } }, "articles": [ { "path": "content/docs/iac/concepts/stacks/_index.md", "url": "/docs/iac/concepts/stacks/", "slug": "docs-iac-concepts-stacks", "lane": "priority", "mode": "fix", "tier": 1, "no_retire": true, "editable": true, "monthly_visits": 12345, "signals": { "gsc": { "impressions": 15234, "ctr": 0.0205, "opportunity": 0.41, "multiplier": 1.1025, "low_ctr_flag": true }, "feedback": { "yes": 4, "no": 9, "neg_rate": 0.6923, "multiplier": 1.27 } }, "last_reviewed": null, "stale_claims": 1, "stale_claim_markers": [ { "entity_key": "version/pulumi-package", "claim_text": "Package source is saved to `packages` in Pulumi.yaml as of Pulumi 3.157.0.", "verdict": "contradicted", "evidence": "CHANGELOG lists \"Save package source to `packages` in Pulumi.yaml on `package add`\" under 3.163.0, not 3.157.0.", "source": "gh api repos/pulumi/pulumi/releases", "checked_at": "2026-08-15", "unresolved_reviews": 0 } ], "score": 0.91 } ] } ``` - `lane` — `priority` (scored pick) or `manual` (workflow_dispatch override). - `mode` — `fix` (this procedure), `glowup`, or `report`. A `report` queue never reaches you: that lane runs no model (see §Report-only mode). - `editable` — false only on generated trees, which is why they are never in a `fix` queue. Nothing you do should ever need to check it. - `stale_claims` (when present) — count of this page's volatile claims the nightly re-verification found contradicted (see §Claims index below). A non-zero count is why the page jumped the queue. - `stale_claim_markers` (when present) — **those findings in full**, each with the `entity_key`, the `claim_text` (the exact sentence the nightly verifier checked — search the page for it; a marker without one predates 2026-09 and names only the entity), the `verdict`, the `evidence` the nightly verifier recorded, the `source` it reached, and `unresolved_reviews` (how many prior reviews saw this marker and left it unresolved). **These are the highest- priority findings in your queue and you must address every one of them.** The nightly lane has already done the expensive part — it identified the entity, reached an authoritative source, and wrote down what that source says — so start here rather than waiting to see whether your own claim extraction happens to re-derive the same finding. It may not: a page boosted for a contradicted version pin was once reviewed, reported "0 contradicted" across 74 re-extracted claims, and merged an unrelated one-line repair while the flagged bug stayed on master. A marker carrying `"escalated": true` has already survived two reviews unresolved, so it no longer jumps the page to the front of the queue and a human may be looking at it — but it is still a live finding and still yours to resolve if you can. Treat it like any other marker. For each marker, either apply the fix, or establish that the flag was wrong (the nightly verdicts are single-sample and do produce false positives — a synthetic module path for a locally generated SDK was once flagged as a broken import). Then list its `entity_key` in the verdict sentinel's `resolved_claims`. A marker you do not resolve is carried onto the next review with `unresolved_reviews` incremented, and after two such rounds it is escalated for a human — so silently skipping one does not make it go away, it just delays it. (That next review is not tomorrow's: the boost sits out a five-day cooldown after any completed review of the page.) - `no_retire` — when true, retirement must never be proposed for this page. This is the **hard veto** on retirement — honor it regardless of evidence. - `reader_signals` / `signals` — Search Console and feedback-widget figures from the optional reader-signals export; `null` when the export wasn't available for the run (a signal-blind run says so, it never fabricates). These fed the selection score and the composed "Why this page" block — they are selection facts, not review instructions. - If `articles` is empty or `halted` is set, do nothing (the workflow won't invoke you in that case, but be defensive). - `traffic.available: false` is not your problem to fix: the dispatcher's degradation-health lane (`scripts/content-review/signal-health.py`) tracks it — along with pulumi/pulumi-service (console-source) access, the holiday feed, and the nightly claims re-verify — and alerts #docs-ops after a week of continuous degradation. Process articles **sequentially**, one at a time, completing each article's fix set and verdict before starting the next. ## Per-article procedure ### 1. Branch — the workflow owns this You do not create branches. The publish job derives the **exact** branch name from the queue entry's `slug` and your verdict's `retirement` flag — `content-review/<slug>` for a fix, `content-review/retire-<slug>` for a retirement proposal (see below) — and a deterministic pre-check has already skipped the run (with a `skipped` verdict) if an open PR owns either branch, so you never need to check for one. ### 2. Pre-compute (deterministic floor) — the workflow runs this for you The `content-review-article` workflow generates the synthetic whole-file diff and runs the docs-review pre-steps **before invoking you**, exactly as the PR review workflow does. The artifacts are already at the repo root when you start: `.fetched-urls.json`, `.candidate-claims.json`, `.verified-claims.json`, `.vale-findings.json`, `.frontmatter-validation.json`, and `.cross-sibling-discovery.json`. **Read them — do not regenerate them.** For reference (and for local runs outside CI), the exact pipeline the workflow executes — scripts live in `.claude/commands/docs-review/scripts/`: ```bash git diff --no-index /dev/null <path> > .synthetic.patch || true python3 .claude/commands/docs-review/scripts/extract-urls-and-fetch.py \ --patch-file .synthetic.patch --out .fetched-urls.json python3 .claude/commands/docs-review/scripts/extract-claims.py \ --patch-file .synthetic.patch --out .candidate-claims-regex.json python3 .claude/commands/docs-review/scripts/extract-claims-llm.py \ --patch-file .synthetic.patch --changed-files <path> \ --pass atomic --scrutiny standard --out .candidate-claims-llm-1.json python3 .claude/commands/docs-review/scripts/extract-claims-llm.py \ --patch-file .synthetic.patch --changed-files <path> \ --pass holistic --scrutiny standard --out .candidate-claims-llm-2.json python3 .claude/commands/docs-review/scripts/merge-claims.py \ --regex .candidate-claims-regex.json \ --llm .candidate-claims-llm-1.json --llm .candidate-claims-llm-2.json \ --out .candidate-claims.json python3 .claude/commands/docs-review/scripts/verify-claims.py \ --in .candidate-claims.json --fetched-urls .fetched-urls.json \ --out .verified-claims.json python3 .claude/commands/docs-review/scripts/frontmatter-validate.py \ --changed-files <path> --out .frontmatter-validation.json python3 .claude/commands/docs-review/scripts/cross-sibling-discover.py \ --changed-files <path> --out .cross-sibling-discovery.json ``` Run Vale in whole-file mode (no `--pr`) and in **fix mode**: ```bash vale --no-exit --output=JSON <path> > .vale-raw.json 2>/dev/null || echo '{}' > .vale-raw.json python3 .claude/commands/docs-review/scripts/vale-findings-filter.py \ --fix-mode --in .vale-raw.json --out .vale-findings.json || echo '[]' > .vale-findings.json ``` `--fix-mode` is required here and is the one place it is used. Without it the filter applies the pinned review's comment budget — 10 findings per file, 50 total, advisory findings de-duplicated per (category, message) — which truncates the backlog you are here to clear. A single-file run hits the per-file cap exactly, you fix those 10, and the next review run surfaces the next 10 as fresh PR churn. The dedup is worse for a fixer than for a reader: `Pulumi.NarrativeWe` emits the same message for every "we will" in the file, so a deduped list shows one line and hides the rest. If any artifact is missing or carries an `errors` field, continue with the artifacts you have and say so in the PR description; never fabricate artifact contents. (Consult flag names with `--help` if running a script by hand — do not guess alternate flags.) ### 3. Triage and fix — HIGH-CONFIDENCE ONLY Read the artifacts and triage per `docs-review:references:pre-computation`'s contract (scripts find facts; you make editorial judgments). The bar for **applying** a fix is strict. Apply only: - **Contradicted claims with an unambiguous correction** — `.verified-claims.json` verdict `contradicted` or `mismatch`, where the authoritative source states the correct value outright (a version number, a price, a flag name). If the correction requires interpretation, do not apply it. - **Dead or redirected internal links** — fix to the canonical full path (`/docs/...`, never `../`). - **Frontmatter violations** from `.frontmatter-validation.json` (broken menu parent, alias collision). - **Deterministic-fix Vale findings** — findings in `.vale-findings.json` stamped `deterministic_fix: true`. The stamp (from the allowlist in `.claude/commands/docs-review/scripts/vale-deterministic-fixes.yaml` — fixed substitutions, canonical spelling, the closed-set cross-reference heading rename) means Vale already knows the exact replacement, so you don't have to author one. It does **not** mean apply blindly. For each, apply the replacement named in the finding's `message` **only after confirming it preserves meaning in this context** — read the surrounding sentence/section. Skip and record under "Findings not applied" (with the reason) when the swap would change meaning, e.g. `click`→`select` inside "click event", a product rename inside a historical quote, or a `See also`/`Related` block whose links are actually sequential (that wants **Next steps**, not the rule's default **Learn more** — apply the correct heading or flag it, don't mislabel intent). This is verify-a-proposed-fix, not compose-a-fix; it is still a judgment. Vale findings *without* the flag are style/judgment nags — leave them for the human (see `docs-review:references:spelling-grammar`). **Never apply a readthrough finding in this lane**, whatever its `fix_class`. Readthrough repairs change a page's structure, and one page read in isolation doesn't tell you why it's shaped the way it is. In September 2026, 4 of the first 14 decided fix-lane PRs carrying one went wrong over the readthrough edit: it broke rendering, flattened prescriptive guidance, or deleted a section's only example. Every readthrough finding is banked for the glow-up lane (see §Glow-up mode), where a human reviews the whole page. `scripts/content-review/publish-gate.py` refuses a `fixed` verdict whose `applied[]` lists a `readthrough` entry, so a readthrough edit fails the whole run. Everything else — `unverifiable` verdicts, low-confidence corrections, prose-quality findings, structural suggestions, **every readthrough finding** (`local_repair` and `reconception` alike), anything you'd phrase with "consider" — goes in the PR description's **Findings not applied** section (one line of reasoning each), not in the diff. That list is the almost-made-the-cut record the human reviewer adjudicates, and the backlog the glow-up lane executes. Count every one of them in the verdict's `skipped_findings`, **including on a `clean` verdict**: the glow-up selector ranks pages on that number. When you flag a `reconception`, set `clarity_flag: true` in the verdict sentinel (step 8) so the ledger carries the signal even on an otherwise-clean page. Record every fix you apply as an entry in the verdict sentinel's `applied` array (step 8): its category, file, **pre-fix** line range, and a pointer to the artifact finding it implements. The publish job deterministically verifies that every hunk in your exported changes falls within the line range of a recorded finding (`scripts/content-review/verify-fix-scope.py`); an edit outside the recorded findings fails that gate and nothing is pushed — so if a change doesn't trace to a finding above, don't make it. #### Re-run Vale to a fixpoint — glow-up lane only **Only when `mode` is `glowup`.** After applying your Vale fixes, re-run the filter over the modified file and triage what comes back, up to **3 rounds total** or until a round yields nothing you would apply: ```bash vale --no-exit --output=JSON <path> > .vale-round<N>.json 2>/dev/null || echo '{}' > .vale-round<N>.json python3 .claude/commands/docs-review/scripts/vale-findings-filter.py \ --fix-mode --in .vale-round<N>.json --out .vale-findings-round<N>.json \ || echo '[]' > .vale-findings-round<N>.json ``` A fix can *create* a finding: Vale's rules are independent, so a phrase one rule steers you toward can be a phrase another rule flags. The case that motivated this loop was #21456 — `write-good.TooWordy` flagged "a number of", the fix chose "several", and `write-good.Weasel` flagged "several". That specific pair is fixed at the source (`Weasel`'s quantifier tokens were removed; see #21470), but nothing prevents the next one, and a page that oscillates should stop here rather than at the reader. Rules for the loop: - **Stop at 3 rounds** and record any still-open findings under **Findings not applied**, with the round count. Non-convergence is a style-config bug worth a human's attention — do not keep grinding, and do not oscillate a phrase back to a form an earlier round already rejected. - **If two rules disagree on the same span, change neither.** Record it under Findings not applied naming both rules. Picking a third phrasing to dodge both is how prose gets worse one round at a time ("provides a number of options" → "provides several options" → "provides options" lost the reader the fact that there are several). - **Respect the churn budget.** `verify-glowup-scope.py` fails the review over `MAX_CHANGED_LINES` (400) total added+deleted. If a later round would push you near it, stop and flag the remainder. - **Keep the round files dot-prefixed and never overwrite `.vale-findings.json`.** The export step stages with `git add -A -- ':!.*' ':!.*/**'`, so a dot-path can't ride along in the handoff patch; a round file named without the dot would land in the diff and fail the scope gate on paths. The step-2 artifact stays the immutable record of what the pre-step found. **Do not do this on the fix lane** (`mode` other than `glowup`), where it would fail the publish gate rather than help. `verify-fix-scope.py` reads its allowed ranges from the `review-snapshot` artifact, which is uploaded *before* the model step and is immutable from then on, and matches hunks against **pre-fix** line numbers. A round-2 finding exists in neither: it is absent from the snapshot, and its line numbers index the already-modified file. Fixes traceable only to a later round therefore read as out-of-scope edits, and the publish job fails before anything is pushed. The glow-up lane is exempt because