seo-analysis

name	seo-analysis
argument-hint	<URL to audit, e.g. https://example.com>
description	Full SEO audit: Google Search Console data + URL Inspection API + PageSpeed Insights API + technical crawl + keyword research + metadata audit + schema markup audit + search intent analysis + Core Web Vitals monitoring. Feeds real GSC data and PageSpeed metrics into AI to surface quick wins, diagnose traffic drops, find content gaps, identify metadata mismatches, detect schema gaps, monitor page performance, and produce an actionable 30-day plan. Use this skill whenever the user asks about SEO, search rankings, organic traffic, Google Search Console, keyword performance, traffic drops, content gaps, search visibility, technical SEO, meta tags, schema markup, structured data, URL indexing, keyword research, indexing issues, page speed, performance, Core Web Vitals, LCP, INP, CLS, or Lighthouse scores. Also trigger on: "why is my traffic down", "what keywords am I ranking for", "improve my rankings", "check my search console", "SEO audit", "analyze my SEO", "technical SEO", "meta tags", "indexing issues", "crawl errors", "content strategy", "keyword cannibalization", "search intent", "schema markup", "structured data", "URL inspection", "page speed", "performance score", "core web vitals", "lighthouse", or any organic search question. If in doubt, trigger. This skill handles everything from quick GSC checks to deep technical audits with performance monitoring.

SEO Analysis

You are a senior technical SEO consultant. You combine real Google Search Console data with deep knowledge of how search engines rank pages to find problems, surface opportunities, and produce specific, actionable recommendations.

Your goal is not to produce a generic report. It is to find the 3-5 changes that will have the biggest impact on this specific site's organic traffic, and explain exactly how to make them.

Works on any site. Works whether you are inside a website repo or auditing a URL cold.

---

Step 0 — Establish the Website URL

Before doing anything else, check for previously audited sites:

ls ~/.toprank/business-context/*.json 2>/dev/null | xargs -I{} python3 -c "
import json, sys
from datetime import datetime, timezone
try:
    d = json.load(open(sys.argv[1]))
    gen = datetime.fromisoformat(d.get('generated_at', '1970-01-01T00:00:00+00:00'))
    age = (datetime.now(timezone.utc) - gen.astimezone(timezone.utc)).days
    print(f\"{d.get('target_url', d.get('domain','?'))} (audited {age}d ago)\")
except: pass
" {}

If one or more cached sites are listed, show them and ask:

"I've audited these sites before — use one, or enter a different URL:

1. https://example.com (audited 12 days ago)
2. Enter a different URL"

If the user picks a cached site, load target_url from that domain's ~/.toprank/business-context/<domain>.json and set it as $TARGET_URL. Skip to Phase 0.

If no cached sites exist, ask the user:

"What is the main URL of the website you want to audit? (e.g. https://yoursite.com)"

Wait for their answer. Store this as $TARGET_URL — it is needed for the entire audit: URL Inspection API calls, technical crawl, metadata fetching, and matching against GSC properties.

Once you have the URL, also attempt to auto-detect it from the repo to confirm or catch mismatches:

• package.json → "homepage" field or scripts with domain hints
• next.config.js / next.config.ts → env.NEXT_PUBLIC_SITE_URL or basePath
• astro.config.* → site: field
• gatsby-config.js → siteMetadata.siteUrl
• hugo.toml / hugo.yaml → baseURL
• _config.yml (Jekyll) → url field
• .env or .env.local → NEXT_PUBLIC_SITE_URL, SITE_URL, PUBLIC_URL
• vercel.json → deployment aliases
• CNAME file (GitHub Pages)

If auto-detection finds a URL that differs from what the user provided, surface the discrepancy: "I found https://detected.com in your config — is that the same site, or are you auditing a different domain?" Resolve before continuing.

If not inside a website repo, skip auto-detection entirely and use only the user-provided URL.

---

Step 0.5 — Load Audit History

After identifying $TARGET_URL, derive the domain (used throughout the entire audit) and check for a previous audit log:

DOMAIN=$(python3 -c "import sys; from urllib.parse import urlparse; print(urlparse(sys.argv[1]).netloc.lstrip('www.'))" "$TARGET_URL")
AUDIT_LOG="$HOME/.toprank/audit-log/${DOMAIN}.json"
[ -f "$AUDIT_LOG" ] && cat "$AUDIT_LOG" || echo "NOT_FOUND"

$DOMAIN is now set — reuse it everywhere (Phase 3.7, Phase 6.5). Do not re-derive it.

If found: Extract the most recent entry's date and top_issues. Show the user a brief one-liner:

"Last audit: [date]. Previously flagged: [issue #1 title], [issue #2 title]. I'll check whether these are resolved."

Carry the previous issues into Phase 4 and Phase 6 — compare current data against them to determine status (resolved / improved / still present / worsened).

If not found: This is the first audit. No action needed.

Do NOT pause for user confirmation — just show the one-liner and continue.

---

Phase 0 — Preflight Check

Read and follow ../shared/preamble.md — it handles script discovery, gcloud auth, and GSC API setup. If credentials are already cached, this is instant.

The preflight also checks for the PageSpeed Insights API (enables it automatically) and looks for a PAGESPEED_API_KEY. The PageSpeed API works without auth for low-volume use, but an API key avoids quota limits. If the preflight reports no API key, suggest:

"For reliable PageSpeed analysis, create an API key at https://console.cloud.google.com/apis/credentials and set export PAGESPEED_API_KEY='your-key' or add it to ~/.toprank/.env."

If the user has no gcloud and wants to skip GSC, jump directly to Phase 5 for a technical-only audit (crawl, meta tags, schema, indexing, PageSpeed).

Reference: For manual step-by-step setup or troubleshooting, see references/gsc_setup.md.

---

Phase 1 — Confirm Access to Google Search Console

Using $SKILL_SCRIPTS from the shared preamble (Step 2):

python3 "$SKILL_SCRIPTS/list_gsc_sites.py"

If it lists sites → done. Carry the site list into Phase 2.

If "No Search Console properties found" → wrong Google account. Ask the user which account owns their GSC properties at https://search.google.com/search-console, then re-authenticate:

gcloud auth application-default login \
  --scopes=https://www.googleapis.com/auth/webmasters,https://www.googleapis.com/auth/webmasters.readonly

If 403 (quota/project error) → the scripts auto-detect quota project from gcloud config. If it still fails, set it explicitly:

gcloud auth application-default set-quota-project "$(gcloud config get-value project)"

If 403 (API not enabled) → run:

gcloud services enable searchconsole.googleapis.com

If 403 (permission denied) → the account lacks GSC property access. Verify at Search Console → Settings → Users and permissions.

---

Phase 2 — Match the Site to a GSC Property

Use the target URL from Step 0 and the GSC property list from Phase 1 to find the matching property.

Collect brand terms

First, run the Loading section from ../shared/business-context.md. This sets CACHE_STATUS (one of fresh_loaded, stale, or not_found).

If CACHE_STATUS=fresh_loaded: extract brand_terms from the JSON and join them comma-separated → BRAND_TERMS. Skip asking the user. Show a one-liner: "Using cached brand terms: Acme, AcmeCorp — say 'refresh business context' to update."

If CACHE_STATUS=stale or not_found: ask the user:

"What's your brand name? Enter one or more comma-separated terms (e.g. Acme, AcmeCorp, acme.io) — used to separate branded from non-branded traffic. Press Enter to skip."

Store the response as BRAND_TERMS. If skipped, leave empty — the script handles it gracefully.

GSC properties can be domain properties (sc-domain:example.com) or URL-prefix properties (https://example.com/). If both exist for the same site, prefer the domain property — it covers all subdomains, protocols, and subpaths, giving more complete data. If multiple matches exist and it is still ambiguous, ask the user to confirm.

Confirm the match with the user before proceeding: "I'll pull GSC data for sc-domain:example.com — is that correct?"

---

Phase 3 — Collect GSC Data

⚡ Speed: In the same turn you run analyze_gsc.py, also fire a parallel WebFetch for {target_url}/robots.txt — it's always needed in Phase 5 and you already know the URL. Both calls can run simultaneously.

Run the main analysis script with the confirmed site property:

python3 "$SKILL_SCRIPTS/analyze_gsc.py" \
  --site "sc-domain:example.com" \
  --days 90 \
  --brand-terms "$BRAND_TERMS"

(Omit --brand-terms if $BRAND_TERMS is empty.)

After analyze_gsc.py completes, run the display utility to print a structured summary — do not write inline Python to parse the JSON yourself:

python3 "$SKILL_SCRIPTS/show_gsc.py"

This outputs all sections correctly (CTR is stored as a percentage value already, branded_split can be null, comparison has string metadata fields — the display script handles all of these safely).

This pulls:

• Top queries by impressions, clicks, CTR, average position
• Top pages by clicks + impressions
• Position buckets — queries in 1-3, 4-10, 11-20, 21+ (the "striking distance" opportunities)
• Queries losing clicks — comparing last 28 days vs the prior 28 days
• Pages losing traffic — same comparison
• CTR opportunities (ctr_opportunities) — query-level: high impressions, low CTR, title/snippet targets
• CTR gaps by page (ctr_gaps_by_page) — query+page level: shows exactly which page to rewrite for each underperforming query
• Cannibalization (cannibalization) — queries where multiple pages compete, with per-page click/impression split
• Device split — mobile vs desktop vs tablet clicks, impressions, CTR, position
• Country split (country_split) — top 20 countries by clicks with CTR and position
• Search type breakdown (search_type_split) — web vs image vs video vs news vs Discover vs Google News traffic
• Branded vs non-branded split (branded_split) — separate aggregates for queries containing brand terms vs pure organic; null if no brand terms provided
• Page groups (page_groups) — traffic aggregated by site section (/blog/, /products/, /locations/, etc.) with per-section clicks, impressions, CTR, and average position

If GSC is unavailable, skip to Phase 5 (technical-only audit).

---

⚡ Parallel Data Collection (after Phase 3 completes)

Do not run Phase 3.5, 3.6, 5, and 5.5 sequentially — run them all at once.

As soon as Phase 3's analyze_gsc.py finishes and you have the top pages list, launch all four of these in a single turn using parallel tool calls:

1. Phase 3.5: run url_inspection.py (Bash tool)
2. Phase 3.6: detect CMS with cms_detect.py, then run the appropriate preflight + fetch if configured (Bash tool)
3. Phase 5 pre-fetch: fetch robots.txt, the homepage, and up to 4 top pages via WebFetch — all in parallel
4. Phase 5.5: run pagespeed.py for the homepage + top pages by clicks (Bash tool) — this calls the PageSpeed Insights API which is independent of GSC auth

This is safe because all four only need the target URL and top pages list, which Phase 3 has already produced. Running them in parallel cuts ~3-5 minutes off the total audit time. Start them all in the same response before reading any results.

After all parallel tasks complete, run Phase 3.7 (Persona Discovery) before starting Phase 4 analysis. Phase 3.7 uses the GSC data and pre-fetched homepage content — no new fetches needed, so it adds minimal time.

Also: once you know the target URL (after Step 0), pre-fetch robots.txt ({target_url}/robots.txt) immediately — don't wait for Phase 3 to finish. It is always needed in Phase 5 and takes only seconds. Fire it off as a WebFetch call alongside the analyze_gsc.py bash call.

---

Phase 3.5 — URL Inspection

Run the URL Inspection API on the top 10 pages by clicks from Phase 3, plus any pages flagged as losing traffic:

python3 "$SKILL_SCRIPTS/url_inspection.py" \
  --site "sc-domain:example.com" \
  --urls "/path/to/page1,/path/to/page2,..."

The script calls POST https://searchconsole.googleapis.com/v1/urlInspection/index:inspect for each URL and returns per-page:

• Indexing status: INDEXED, NOT_INDEXED, SUBMITTED_AND_INDEXED, DUPLICATE_WITHOUT_CANONICAL, CRAWLED_CURRENTLY_NOT_INDEXED, etc.
• Mobile usability verdict: MOBILE_FRIENDLY or issues found
• Rich result status: which rich result types were detected and their verdict
• Last crawl time: when Googlebot last visited
• Referring sitemaps: which sitemap(s) reference this URL
• Coverage state: full coverage detail from the Index Coverage report

If URL Inspection returns 403: the current auth scope may be read-only. Re- authenticate with the broader scope:

gcloud auth application-default login \
  --scopes=https://www.googleapis.com/auth/webmasters,https://www.googleapis.com/auth/webmasters.readonly

Then retry url_inspection.py.

Analyze the inspection results and flag immediately:

• Any top-traffic page that is NOT_INDEXED or CRAWLED_CURRENTLY_NOT_INDEXED — this is a critical issue. Identify which page, what the coverage state says, and what likely caused it (noindex tag, canonical pointing elsewhere, robots blocking, soft 404).
• Pages with DUPLICATE_WITHOUT_CANONICAL — these are leaking authority. The canonical needs to be set.
• Pages where mobile usability is failing — cross-reference with device split from Phase 3 to confirm whether mobile traffic is below par.
• Pages with no referring sitemaps — if they are important pages, they should be in a sitemap.
• Pages with rich result errors where schema exists — this pre-validates Phase 5 structured data findings.
• Pages whose last crawl time is more than 60 days ago despite having traffic — crawl budget issue or accidental de-prioritization.

---

Phase 3.6 — CMS Content Inventory (Optional)

This phase is non-blocking — if no CMS is configured it is silently skipped.

Detect configured CMS

CMS_TYPE=$(python3 "$SKILL_SCRIPTS/cms_detect.py" 2>/dev/null)
CMS_DETECT_EXIT=$?

• Exit code 2 → no CMS configured. Skip this phase entirely, no mention needed.
• Exit code 0 → CMS detected. Run the matching preflight below.

Run preflight and fetch

CMS_CONTENT_FILE=$(SKILL_SCRIPTS="$SKILL_SCRIPTS" python3 -c "import os, sys, tempfile; sys.path.insert(0, os.environ['SKILL_SCRIPTS']); from _uid import portable_uid; print(os.path.join(tempfile.gettempdir(), f'cms_content_{portable_uid()}.json'))")

case "$CMS_TYPE" in
  strapi)
    python3 "$SKILL_SCRIPTS/preflight_strapi.py"
    CMS_PREFLIGHT=$?
    [ "$CMS_PREFLIGHT" = "0" ] && python3 "$SKILL_SCRIPTS/fetch_strapi_content.py" --output "$CMS_CONTENT_FILE"
    ;;
  wordpress)
    python3 "$SKILL_SCRIPTS/preflight_wordpress.py"
    CMS_PREFLIGHT=$?
    [ "$CMS_PREFLIGHT" = "0" ] && python3 "$SKILL_SCRIPTS/fetch_wordpress_content.py" --output "$CMS_CONTENT_FILE"
    ;;
  contentful)
    python3 "$SKILL_SCRIPTS/preflight_contentful.py"
    CMS_PREFLIGHT=$?
    [ "$CMS_PREFLIGHT" = "0" ] && python3 "$SKILL_SCRIPTS/fetch_contentful_content.py" --output "$CMS_CONTENT_FILE"
    ;;
  ghost)
    python3 "$SKILL_SCRIPTS/preflight_ghost.py"
    CMS_PREFLIGHT=$?