llm-seo-readiness

// Validates Aptos docs changes for LLM consumption (llms.txt, curated feeds, Markdown exports, HTML→Markdown) and for SEO (metadata, sitemap, robots, structured data). Use when adding or restructuring docs, editing llms routes or curation, changing Head/meta/OG, updating robots.txt or sitemap behavior, or when the user mentions LLMs.txt, AI tool ingestion, crawlers, discoverability, or SEO.

name	llm-seo-readiness
description	Validates Aptos docs changes for LLM consumption (llms.txt, curated feeds, Markdown exports, HTML→Markdown) and for SEO (metadata, sitemap, robots, structured data). Use when adding or restructuring docs, editing llms routes or curation, changing Head/meta/OG, updating robots.txt or sitemap behavior, or when the user mentions LLMs.txt, AI tool ingestion, crawlers, discoverability, or SEO.

LLM and SEO readiness (Aptos docs)

LLM / machine-readable

Frontmatter: Every new or renamed doc needs accurate title and description in English; they surface in /llms.txt and .md exports. Mirror updates in src/content/docs/zh/ per agent guidelines (Spanish es/ is out of scope for agent-maintained docs).
Curated lists: If a page should appear in the LLM index or early in full corpus, update src/lib/llms-curated-ids.ts (LLMS_INDEX_SECTIONS, LLMS_SMALL_DOC_IDS, FULL_PRIORITY_DOC_IDS as appropriate). Build fails if an id is missing from English non-draft docs.
Index copy: User-facing explanations live in src/content/docs/llms-txt.mdx, src/content/docs/build/ai.mdx, and the Chinese zh/ counterparts—keep URLs and feed names aligned with src/pages/llms-index.ts.
HTML → Markdown: Shared logic is src/lib/llms-html-sanitize.ts. When minifying for llms-small.txt, collapse spaces/tabs only—never all \s (newlines must survive for fenced code and Markdown structure).
Route wiring: Custom handlers are swapped in via src/integrations/llms-txt-index.ts; endpoint implementations live under src/pages/llms-index.ts, src/pages/llms-small.txt.ts, src/pages/llms-full.txt.ts.
Robots: public/robots.txt should stay consistent with sitemap URL and, when feeds change, the commented LLMs.txt pointers at the bottom. Keep the Content-Signal line intact so AI crawlers see the same preferences the rest of the ecosystem uses.
Agent discovery surface: .well-known/api-catalog, .well-known/mcp/server-card.json, .well-known/agent-skills/index.json, the global Link header in vercel.json, and the Head override's <link rel="…"> tags must advertise the same URLs. Re-run pnpm test tests/agent-discovery.test.ts after any change and see the Agent discovery & readiness section of CLAUDE.md for the full checklist.

SEO

Per-page metadata: Starlight frontmatter title / description feed OG, Twitter, and schema in src/starlight-overrides/Head.astro—avoid empty or placeholder descriptions on public pages.
Sitemap: Produced by @astrojs/sitemap in astro.config.mjs; ensure new top-level routes or major URL changes still make sense for indexing.
Hreflang / alternates: Head override builds language alternates from SUPPORTED_LANGUAGES—new locales need config updates, not only content folders.
Draft / hidden content: Do not rely on curated LLM ids or public sitemap for content that must stay off production; follow existing draft filtering in the .md pipeline and curation tests.

Verification

After substantive changes:

pnpm test tests/llms-curated-ids.test.ts tests/llms-html-sanitize.test.ts tests/agent-discovery.test.ts
pnpm lint && pnpm check

For full coverage: pnpm test and a production pnpm build when touching routes or integrations.

File map

Area	Primary locations
Curation	`src/lib/llms-curated-ids.ts`, `src/lib/llms.ts`
llms.txt body	`src/pages/llms-index.ts`
Feed endpoints	`src/pages/llms-small.txt.ts`, `src/pages/llms-full.txt.ts`
Plugin override	`src/integrations/llms-txt-index.ts`
Markdown export / sanitize	`src/lib/llms-html-sanitize.ts`, `src/pages/[...slug].md.ts`
User docs	`src/content/docs/llms-txt.mdx`, `build/ai.mdx`, `zh/`
SEO head	`src/starlight-overrides/Head.astro`
Crawlers / sitemap hint	`public/robots.txt`
Agent discovery — well-known	`public/.well-known/api-catalog`, `public/.well-known/mcp/server-card.json`, `public/.well-known/agent-skills/index.json`, `public/.well-known/oauth-protected-resource`, `public/.well-known/openid-configuration`, `public/.well-known/oauth-authorization-server`
Agent discovery — Link header	`vercel.json` (global `/(.*)` entry)
Markdown negotiation + WebMCP	`src/middlewares/markdown-negotiation.ts`, `src/scripts/webmcp-register.ts`, `src/types/webmcp.d.ts`
Agent discovery regression tests	`tests/agent-discovery.test.ts`

LLM and SEO readiness (Aptos docs)

LLM / machine-readable

Frontmatter: Every new or renamed doc needs accurate title and description in English; they surface in /llms.txt and .md exports. Mirror updates in src/content/docs/zh/ per agent guidelines (Spanish es/ is out of scope for agent-maintained docs).

Curated lists: If a page should appear in the LLM index or early in full corpus, update src/lib/llms-curated-ids.ts (LLMS_INDEX_SECTIONS, LLMS_SMALL_DOC_IDS, FULL_PRIORITY_DOC_IDS as appropriate). Build fails if an id is missing from English non-draft docs.

Index copy: User-facing explanations live in src/content/docs/llms-txt.mdx, src/content/docs/build/ai.mdx, and the Chinese zh/ counterparts—keep URLs and feed names aligned with src/pages/llms-index.ts.

HTML → Markdown: Shared logic is src/lib/llms-html-sanitize.ts. When minifying for llms-small.txt, collapse spaces/tabs only—never all \s (newlines must survive for fenced code and Markdown structure).

Route wiring: Custom handlers are swapped in via src/integrations/llms-txt-index.ts; endpoint implementations live under src/pages/llms-index.ts, src/pages/llms-small.txt.ts, src/pages/llms-full.txt.ts.

Robots: public/robots.txt should stay consistent with sitemap URL and, when feeds change, the commented LLMs.txt pointers at the bottom. Keep the Content-Signal line intact so AI crawlers see the same preferences the rest of the ecosystem uses.

Agent discovery surface: .well-known/api-catalog, .well-known/mcp/server-card.json, .well-known/agent-skills/index.json, the global Link header in vercel.json, and the Head override's <link rel="…"> tags must advertise the same URLs. Re-run pnpm test tests/agent-discovery.test.ts after any change and see the Agent discovery & readiness section of CLAUDE.md for the full checklist.

SEO

Per-page metadata: Starlight frontmatter title / description feed OG, Twitter, and schema in src/starlight-overrides/Head.astro—avoid empty or placeholder descriptions on public pages.

Sitemap: Produced by @astrojs/sitemap in astro.config.mjs; ensure new top-level routes or major URL changes still make sense for indexing.

Hreflang / alternates: Head override builds language alternates from SUPPORTED_LANGUAGES—new locales need config updates, not only content folders.

Draft / hidden content: Do not rely on curated LLM ids or public sitemap for content that must stay off production; follow existing draft filtering in the .md pipeline and curation tests.

Verification

After substantive changes:

pnpm test tests/llms-curated-ids.test.ts tests/llms-html-sanitize.test.ts tests/agent-discovery.test.ts pnpm lint && pnpm check

For full coverage: pnpm test and a production pnpm build when touching routes or integrations.

File map

Area

Primary locations

Curation

src/lib/llms-curated-ids.ts, src/lib/llms.ts

llms.txt body

src/pages/llms-index.ts

Feed endpoints

src/pages/llms-small.txt.ts, src/pages/llms-full.txt.ts

Plugin override

src/integrations/llms-txt-index.ts

Markdown export / sanitize

src/lib/llms-html-sanitize.ts, src/pages/[...slug].md.ts

User docs

src/content/docs/llms-txt.mdx, build/ai.mdx, zh/

SEO head

src/starlight-overrides/Head.astro

Crawlers / sitemap hint

public/robots.txt

Agent discovery — well-known

public/.well-known/api-catalog, public/.well-known/mcp/server-card.json, public/.well-known/agent-skills/index.json, public/.well-known/oauth-protected-resource, public/.well-known/openid-configuration, public/.well-known/oauth-authorization-server

Agent discovery — Link header

vercel.json (global /(.*) entry)

Markdown negotiation + WebMCP

src/middlewares/markdown-negotiation.ts, src/scripts/webmcp-register.ts, src/types/webmcp.d.ts

Agent discovery regression tests

tests/agent-discovery.test.ts