docs-seo-audit

# SEO Audit Crawl the live docs.warp.dev sitemap to find SEO issues and fix them in the source files. ## Agent-doc quality contract Any PR this skill opens follows the shared v1 agent-doc quality contract in `.agents/references/doc-quality-policy.md`: apply the `warpy-factory` label and add the `## Documentation risk` block (`.agents/skills/doc_quality_policy/finalize_pr_contract.py build`). Title, description, and metadata fixes that preserve product meaning are typically `low` risk under the allowlist. ## Running the audit From the docs repo root: ```bash python3 .agents/skills/docs-seo-audit/scripts/seo_audit.py \ --repo-root . \ --output /tmp/seo-report.json ``` The script fetches all pages listed in `https://docs.warp.dev/sitemap-index.xml` (a sitemap index including sub-sitemaps), extracts SEO metadata from each page's HTML, and checks for issues. ### Options - `--repo-root PATH` — Path to the docs repo root. Enables mapping live URLs to local source files in the report. - `--output FILE` — Write the JSON report to a file (otherwise prints to stdout). - `--max-pages N` — Limit to the first N pages (useful for quick testing). - `--workers N` — Number of concurrent fetch workers (default: 6). ### Quick test with a small sample ```bash python3 .agents/skills/docs-seo-audit/scripts/seo_audit.py --repo-root . --max-pages 20 --output /tmp/seo-sample.json ``` ## Reading the report The JSON report contains: - `total_pages` — Number of pages scanned - `total_issues` — Total issues found - `by_severity` — Counts by severity (`error`, `warning`, `info`) - `by_type` — Counts by issue type - `issues` — Array of individual issues, sorted by severity Each issue includes: - `url` — The live page URL - `source_file` — The local source file (if `--repo-root` was provided) - `severity` — `error` (must fix), `warning` (should fix), or `info` (nice to fix) - `type` — Issue category (see below) - `message` — Human-readable description ### Issue types **Errors (must fix)**: - `duplicate_title` — Multiple pages share the same `<title>` tag. Hurts search rankings and confuses users. - `missing_title` — Page has no `<title>` tag. - `fetch_error` — Page could not be fetched. **Warnings (should fix)**: - `duplicate_description` — Multiple pages share the same meta description. - `missing_description` — Page has no meta description. Search engines may generate a snippet from page content instead. - `title_too_short` — Title under 20 chars; may not be descriptive enough for search results. - `title_too_long` — Title over 70 chars; Google will truncate it in search results. - `missing_h1` — No H1 heading found on the page. - `multiple_h1` — More than one H1 heading on the page. **Info (nice to fix)**: - `description_too_short` — Description under 50 chars. - `description_too_long` — Description over 160 chars; will be truncated in search results. - `og_title_mismatch` — OG title differs from the title tag (usually auto-matched by Astro Starlight). - `og_description_mismatch` — OG description differs from the meta description. ## Reporting results After running the audit, ALWAYS report the results to the user before taking any action. Include: 1. **Summary**: Total pages scanned, total issues, breakdown by severity (errors / warnings / info) 2. **Errors first**: List every error-severity issue with the URL, source file, and what's wrong. These are the most impactful and should be fixed. 3. **Warnings**: Summarize warning-severity issues grouped by type (e.g., "58 pages missing meta descriptions"). List specific pages only for the most impactful ones (duplicate titles, duplicate descriptions). 4. **Info**: Briefly note info-level issues with counts. No need to list every page. 5. **If no issues found**: Explicitly tell the user everything looks clean — don't just silently finish. Example report format: ``` SEO audit complete: 276 pages scanned, 183 issues found. **Errors (8):** - Duplicate title "Overview | Agents | Warp" on 4 pages: - /agent-platform (source: src/content/docs/agent-platform/index.mdx) - /agent-platform/local-agents/overview (source: src/content/docs/agent-platform/local-agents/overview.mdx) ... **Warnings (108):** - 58 pages missing meta descriptions (top offenders: ...) - 33 titles too short (<20 chars) - 15 titles too long (>70 chars) **Info (67):** - 55 descriptions over 160 chars - 12 descriptions under 50 chars ``` After reporting, ask the user which issues they want to fix before making changes. ## Fixing issues Before making any changes, read these references: 1. `references/starlight-seo.md` in this skill directory — explains the non-obvious way Astro Starlight generates title tags from sidebar config label text (not the H1 heading). 2. `AGENTS.md` at the docs repo root — the single source of truth for documentation style, voice, terminology, and formatting. All titles and descriptions you write must follow these conventions (terminology, sentence case, active voice, frontmatter description format). ### Key principles 1. **Title tags come from the frontmatter `title` field if present, otherwise from sidebar config labels in `src/sidebar.ts`** — not the H1 heading. To fix a title without changing the sidebar label, add or update the `title` field in the page's frontmatter (and optionally add `sidebar.label` to preserve the short nav label). To fix a title by changing the nav label, change the `label` property in `src/sidebar.ts`. For pages listed as bare slugs (e.g., `'terminal/windows/tabs'`), add an explicit `{ slug, label }` object to override the label. 2. **Meta descriptions come from frontmatter**. To fix a description, edit the `description:` field in the page's YAML frontmatter. 3. **OG and Twitter tags mirror title and description** automatically. No separate fix needed. 4. **Changing a sidebar config label has side effects**: it also changes the sidebar label, breadcrumbs, and prev/next pagination. URLs are NOT affected (URLs are based on the file path/slug). 5. **Never add or edit a manual H1 heading in a markdown file.** Astro Starlight auto-generates the page's H1 from the frontmatter `title` (falling back to the sidebar label or slug, per the precedence in `references/starlight-seo.md`), so this repo's pages never write an explicit `# Heading` at the top of the body — that pattern is leftover behavior from the old GitBook-based docs platform. Changing a `title` or sidebar `label` never requires a matching H1 edit, because there is no H1 in the markdown to edit. If the audit's `multiple_h1` or `missing_h1` check flags a page and you find a manual H1 in its body, treat that as an incidental defect: remove the manual H1 (do not keep it in sync with the title) as its own cleanup, separate from any title/description fix. 6. **All titles, labels, and H1 headings must use sentence case.** Capitalize only the first word and proper nouns. Only recognized proper feature names retain their capitalization — check `AGENTS.md` and `.agents/references/terminology.md` for the canonical list. Common mistakes to avoid: - ✅ `Computer use for agents` — generic term, sentence case - ❌ `Computer Use for Agents` — not a proper feature name, don't capitalize - ✅ `API keys` — generic term, sentence case - ❌ `API Keys` — not a proper feature name - ✅ `Known issues` — generic term, sentence case - ❌ `Known Issues` — not a proper feature name - ✅ `Slack integration` — external product name (Slack) + generic term - ❌ `Slack Integration for Oz` — "Integration" is not a proper feature name; "for Oz" is imprecise (see terminology rules) - ✅ `Agent Mode` — proper feature name, retains capitalization - ✅ `Codebase Context` — proper feature name, retains capitalization - ✅ `Tab Configs` — proper feature name, retains capitalization - ✅ `Warp Drive` — proper feature name, retains capitalization ### Title exceptions Some page titles are intentionally short or specific and must **not** be changed, even if they trigger a `title_too_short` warning. Skip these pages and note them as intentional exceptions in your report: - **`src/content/docs/changelog/index.mdx`** (`Changelog`) — "Changelog" is a clear, universally understood industry term. Branding prefixes like "Warp changelog" or "Release changelog" add no descriptive value and this title should remain as-is. - **`src/content/docs/terminal/appearance/app-icons.mdx`** (`Custom app icons`) — The article explicitly explains that *custom* app icons are not available to users. The sidebar label is intentional and must not be changed. - **`src/content/docs/guides/index.mdx`** (`Guides`) — "Guides" is the landing page for the Guides section. The title is clear and matches the section name; prefixing it (e.g., "Developer Guides") adds no value and creates a mismatch with the section title shown in breadcrumbs. - **`src/content/docs/terminal/windows/tabs.mdx`** (`Tabs`) — The sidebar section header ("Windows and Tabs") already provides terminal context; adding a "Terminal" prefix creates a name not used in the UI or anywhere else in the docs. The `title_too_short` warning is intentionally suppressed — the title is short by design. Do not rename to "Terminal tabs". - **`src/content/docs/terminal/windows/vertical-tabs.mdx`** (`Vertical Tabs`) — Same rationale: the section header disambiguates the terminal context. The `title_too_short` warning is intentionally suppressed. Do not rename to "Terminal vertical tabs". - **`src/content/docs/terminal/windows/split-panes.mdx`** (`Split panes`) — Same rationale: the section header disambiguates the terminal context. The `title_too_short` warning is intentionally suppressed. Do not rename to "Terminal split panes". - **`src/content/docs/terminal/windows/tab-configs.mdx`** (`Tab Configs`) — Same rationale: the section header disambiguates the terminal context. Additionally, "Tab Configs" is a proper feature name and should not be prefixed. The `title_too_short` warning is intentionally suppressed. Do not rename to "Terminal Tab Configs". - **`src/content/docs/terminal/sessions/index.mdx`** (`Sessions`) — The sidebar section header ("Sessions") already provides terminal context. The `title_too_short` warning is intentionally suppressed. Do not rename to "Terminal sessions". - **`src/content/docs/agents/cli/oz-cli/artifacts.mdx`** (`Artifacts`) — The Agents > CLI section provides context, and "Artifacts" matches the `oz artifact` resource that the page documents. The `title_too_short` warning is intentionally suppressed. When the audit flags these pages for `title_too_short`, exclude them from your fix list and include a note in your report explaining they are intentional exceptions. If you believe a new title should be added to this exceptions list, flag it for human review before proceeding. ### Sidebar config labels vs. page titles Sidebar config labels (the `label` property in `src/sidebar.ts`) and frontmatter `title` values are **intentionally different** in some cases. Do not change either to match the other unless you are fixing a genuine duplicate title collision. Specifically: - Do **not** add section-context prefixes (like "Terminal", "Warp", or "Agent") to short but accurate titles just because the title appears generic in isolation. Sidebar context already provides that disambiguation. - Do **not** rename sidebar config labels for pages in the exceptions list above. - Do **not** sync sidebar config label text to match the frontmatter `title` (or vice versa) as a standalone change — the two are allowed to differ. Neither one requires touching an H1 in the markdown body, since there isn't one (see Key principle #5). ### Fixing duplicate titles This is the most impactful issue. Common causes: - Multiple pages with `label: 'Overview'` under different sections — these all produce the same `<title>` tag - Generic labels like `'Getting Started'` or `'FAQs'` repeated across sections There are two ways to fix duplicate titles, depending on whether the short sidebar label is intentional: #### Preferred: frontmatter `title` + `sidebar.label` (keep short nav labels) When a page intentionally uses a short sidebar label like "Overview" because the section header provides context, **do not rename the sidebar config label**. Instead, add a descriptive `title` in the page's frontmatter and use `sidebar.label` to preserve the short nav label: ```yaml --- title: Capabilities overview sidebar: label: "Overview" --- ``` Astro Starlight uses `title` for the `<title>` tag and `sidebar.label` for the sidebar. This gives each page a unique, SEO-friendly title while keeping the sidebar clean. Example: - `agent-platform/capabilities/index.mdx`: `title: 'Capabilities overview'` + `sidebar.label: 'Overview'` - `platform/integrations/index.mdx`: `title: 'Integrations overview'` + `sidebar.label: 'Overview'` Do not add or update an H1 in the markdown file — Starlight renders the page's H1 from this frontmatter `title` automatically (see Key principle #5 above). #### Alternative: rename the sidebar config label If the short label is not intentional, rename the `label` in `src/sidebar.ts` to be unique and descriptive. Use sentence case and correct terminology per `AGENTS.md` (e.g., capitalize proper feature names like "Agent Mode", "Warp Drive", "Codebase Context" — but not generic terms like "overview", "quickstart", or "agents"). Example: - Before: `{ slug: 'agent-platform/local-agents', label: 'Overview' }` + `{ slug: 'platform', label: 'Overview' }` - After: `{ slug: 'agent-platform/local-agents', label: 'Local agents overview' }` + `{ slug: 'platform', label: 'Cloud agents overview' }` Do not add or update an H1 in the markdown file for this — Starlight renders the page's H1 from the frontmatter `title` (or the slug when no `title` is set), never from the sidebar label (see Key principle #5 above). ### Fixing missing descriptions Add a `description` field to the page's frontmatter: ```yaml --- description: >- A concise 1-2 sentence summary of what this page covers and what value it provides to the reader. --- ``` Write descriptions that: - Summarize the page's content in 50-160 characters - Include the primary keyword/topic naturally - Describe user benefit, not just feature existence - Follow the voice and terminology conventions in `AGENTS.md` (e.g., use "Warp" not "Warp Terminal", "Oz" for the orchestration platform, active voice, user-focused phrasing) ### Review loop After making fixes, review every change before presenting to the user. Run through this checklist, and if anything fails, fix it and re-run the checklist from the top. Only present changes when everything passes. - **Does this still mean the same thing?** Titles and descriptions must accurately represent the page content. Read the actual page before writing or rewriting anything. Never invent features, capabilities, or details that aren't on the page. If unsure what the page covers, read it first. - **Did I introduce a new duplicate?** Scan the full sidebar config in `src/sidebar.ts`. Verify every label is unique within the site. This is the most common mistake — fixing one duplicate by picking a name that collides with an existing entry. - **Did I leave any manual H1 behind?** Scan every file you touched for a body-level `# Heading` and remove it — Starlight already renders the page's H1 from frontmatter `title`, so a manual H1 is always a duplicate, never something to add or sync with a title/label change. - **Is the terminology right?** Cross-check against `AGENTS.md` and how the feature is actually referred to in the existing docs. Don't rename things to terms that aren't used elsewhere in the docs. - **Is the casing right?** All labels and H1 headings must use sentence case. Proper product feature names (e.g., "Agent Mode", "Codebase Context", "Admin Panel", "Remote Control", "Warp Drive") retain their capitalization, but generic terms ("overview", "quickstart", "agents", "notifications") are lowercase. Never use title case. - **Does this read naturally in context?** Consider how the title appears (a) as a sidebar label under its section header, and (b) as a search result: `{Title} | {Topic} | Warp`. If it sounds awkward or uses internal jargon that users wouldn't recognize, rephrase. - **Are descriptions grounded in page content?** Don't write descriptions based on the title alone. Read the page, then summarize what's actually there. - **Any other improvements nearby?** Look at adjacent entries in `src/sidebar.ts`. Are there other generic labels ("Overview", "Getting Started", "FAQ") that could become duplicates in the future? Flag them proactively. ### Delivering changes **Interactive (user in the loop):** Present the full list of changes with old → new values so the user can review at a glance. Wait for approval before committing. **Autonomous (scheduled agent, CI):** The review loop above is your quality gate. Commit directly and include the old → new summary in the PR description. **PR conventions:** - Title must be prefixed with `Automated SEO fixes:` (e.g., `Automated SEO fixes: resolve duplicate titles in agent-platform and reference spaces`) - Add the `seo` label to the PR (e.g., `gh pr create --label seo`) ## Slack notification (optional) If instructed to send a report to Slack, post a summary after the audit completes. This works regardless of whether fixes were made. **Post at most once per run.** This mirrors the "Never post twice for one run" rule in `.agents/references/skill-authoring-guidelines.md` and the same-run guard in `weekly-404-monitor`. Revise the summary wording as many times as you like before sending — never after. Once a post attempt succeeds (`ok: true`), treat the notification as terminal for this run: do not re-post to fix a typo, tighten wording, or add a detail you forgot. A second, "cleaned up" post is a worse outcome than an imperfect first one. **Check for an existing same-day post before sending.** Before posting, check whether a top-level message matching `*SEO Audit — <today's date>*` already exists in the target channel — this catches both a rerun of this agent for the same day and a mid-run retry after an apparent failure that actually succeeded. Skip the post if one is found. 1. Check if `BUZZ_SLACK_TOKEN` environment variable exists. 2. If the token exists, compose the summary (see the categorization and format rules below) and send it with the one-shot poster script in the "Sending the notification" section, which performs the dedupe check and the post in a single invocation. **Categorizing issues in the summary:** Before composing the message, cross-reference every issue against the title exceptions list above and check whether the issue has a local source file. Classify each issue into exactly one bucket: - **Fixed** — issues you resolved in this run - **Unfixable** — issues with no local source file - **Allowlisted** — issues that match a title exception entry (these are intentional, not problems) - **Remaining** — everything else (genuine issues that still need attention) Only include a section in the Slack message if its count is > 0. Never list allowlisted pages under "Remaining". **If a PR was opened**, include the PR link and a summary of what was fixed: ``` *SEO Audit — <date>* <total_pages> pages scanned | <total_issues> issues found *Fixed (<count>):* • Duplicate title "Overview | Agents | Warp" → split into 4 unique titles • Trimmed 3 overly long descriptions to ≤160 chars