seo-recovery

# SEO Recovery Protocol ## When This Skill Activates - "SEO traffic dropped", "organic traffic down", "lost rankings" - "Indexing issues", "pages not indexed", "deindexed" - "Canonical URL", "canonical mismatch", "wrong canonical" - "Hreflang", "hreflang errors", "language tags" - "Sitemap", "sitemap errors", "toxic sitemap", "missing sitemap" - "Search Console", "GSC", "crawl errors", "coverage issues" - "Robots.txt", "noindex", "blocked by robots" - "Google not showing my pages" ## Anti-Hallucination Rules (NEVER violate) | Rule | Description | |------|-------------| | **NEVER ASSUME INDEXING STATUS** | Always verify via GSC URL Inspection or `site:` search. Never say "this page is indexed" without checking | | **GSC DATA BEFORE CONCLUSIONS** | Pull actual Search Analytics data before diagnosing. Traffic estimates, guesses, or assumptions are forbidden | | **VERIFY CANONICAL TAGS IN SOURCE** | Always `curl` or read the actual HTML source. Do NOT trust CMS admin panels or visual page checks -- the rendered HTML is ground truth | | **CHECK BOTH HTTP AND HTTPS** | Canonical mismatches often involve protocol differences. Always check both variants | | **DISTINGUISH CRAWLED VS INDEXED** | A page being crawled does NOT mean it is indexed. These are separate states in GSC Coverage | | **HOSTINGER CDN DISTRUST** | After deploying SEO fixes to Hostinger, verify at server level via `curl`. CDN may serve stale content for 5-15 minutes | | **NO BULK CHANGES WITHOUT BACKUP** | Before modifying sitemaps, robots.txt, or canonical tags at scale, document the current state | | **TIMESTAMP YOUR EVIDENCE** | GSC data has processing delays (24-72h). Always note when data was pulled vs when changes were made | --- ## Phase 0: Confirm Scope (MANDATORY - Do This First) Before any investigation: ``` 1. What is the domain? (exact URL, including protocol and www vs non-www) 2. When did the traffic drop start? (exact date or date range) 3. How severe is the drop? (percentage, from X to Y) 4. Is it site-wide or specific pages/sections? 5. Were any changes deployed around that time? (redesign, migration, CMS update, hosting change) 6. Which GSC property are we using? (domain property vs URL prefix?) 7. What is the hosting platform? (Hostinger? Other? -- affects deployment workflow) ``` **If the user cannot answer #2 and #3, start with GSC Search Analytics to establish the timeline.** --- ## Phase 1: SEO Diagnostic Protocol ### Step 1: Establish the Timeline (Search Analytics) Pull traffic data to identify exactly when the drop occurred: ``` GSC MCP QUERY: - Tool: google_searchconsole (via MCP Gateway) - Metric: clicks, impressions, CTR, position - Date range: 90 days (or wider if drop is older) - Dimensions: date (for timeline), page (for affected URLs), query (for lost keywords) ``` **What to look for:** - Sudden cliff (overnight) = algorithmic penalty or technical issue - Gradual decline over weeks = content quality, competition, or slow technical degradation - Specific pages/sections = localized issue (canonical, noindex on section) - All pages equally = site-wide issue (robots.txt, domain-level penalty, DNS) ### Step 2: Check Indexing Status (Coverage Report) Query GSC Coverage to understand indexing state: ``` KEY COVERAGE STATES: - "Valid" = Indexed and serving in search - "Valid with warnings" = Indexed but has issues - "Excluded" = Not indexed (check reason!) - "Error" = Crawl/index errors COMMON EXCLUSION REASONS: - "Alternate page with proper canonical tag" = Canonical pointing elsewhere - "Duplicate without user-selected canonical" = Google chose a different canonical - "Crawled - currently not indexed" = Google crawled but chose not to index - "Discovered - currently not indexed" = Google knows about it but hasn't crawled - "Page with redirect" = URL redirects to another page - "Blocked by robots.txt" = robots.txt is blocking crawlers - "Excluded by 'noindex' tag" = meta noindex or X-Robots-Tag ``` ### Step 3: URL Inspection (Spot-Check Key Pages) For affected pages, run URL Inspection to get Google's view: ``` CHECK THESE FOR EACH AFFECTED PAGE: 1. Is the page indexed? 2. What canonical does Google see? (vs what the page declares) 3. Is it mobile-friendly? 4. Are there any crawl issues? 5. Last crawl date -- is it recent or stale? 6. Does the rendered HTML match expectations? ``` ### Step 4: Root Cause Analysis Based on Phases 1-3, route to the appropriate diagnosis: | Evidence Pattern | Likely Root Cause | Go To | |------------------|-------------------|-------| | Coverage shows "Alternate page with proper canonical" | Canonical URL mismatch | CANONICAL FIX | | Coverage shows "Blocked by robots.txt" | robots.txt blocking | ROBOTS.TXT FIX | | Coverage shows "Excluded by noindex" | Meta noindex tag | NOINDEX FIX | | Many pages "Crawled - currently not indexed" | Quality/thin content or crawl budget | CRAWL BUDGET | | Hreflang errors in Coverage or manual check | Hreflang implementation bugs | HREFLANG FIX | | Sitemap errors in GSC Sitemaps report | Toxic or broken sitemaps | SITEMAP FIX | | Sudden drop + manual action in GSC | Google penalty | MANUAL ACTION | | Drop coincides with site migration/redesign | Migration issues | MIGRATION FIX | --- ## Phase 2: Common SEO Issues & Fixes ### CANONICAL FIX: Canonical URL Mismatches **Diagnosis:** ```bash # Check canonical tag in actual HTML source curl -s "https://example.com/page" | grep -i 'rel="canonical"' curl -s "https://example.com/page" | grep -i "rel='canonical'" # Check HTTP vs HTTPS canonical curl -s "http://example.com/page" | grep -i 'canonical' curl -s "https://example.com/page" | grep -i 'canonical' # Check www vs non-www curl -s "https://www.example.com/page" | grep -i 'canonical' curl -s "https://example.com/page" | grep -i 'canonical' # Check for canonical in HTTP headers (X-Robots-Tag) curl -sI "https://example.com/page" | grep -i 'link.*canonical' # Bulk check across pages (sample 20 URLs from sitemap) curl -s "https://example.com/sitemap.xml" | grep -oP '<loc>\K[^<]+' | head -20 | while read url; do canonical=$(curl -s "$url" | grep -oP 'rel="canonical"[^>]*href="[^"]*"' | grep -oP 'href="\K[^"]+') echo "$url -> $canonical" done ``` **Common mismatch patterns:** - HTTP canonical on HTTPS page (or vice versa) - www canonical on non-www page (or vice versa) - Trailing slash mismatch (`/page` vs `/page/`) - Query parameter pollution (`/page?utm_source=...` as canonical) - Relative URL instead of absolute URL in canonical tag - CMS generating wrong canonical (pagination, filters, search pages) **Fix approach:** 1. Identify the canonical URL pattern that Google should use 2. Update canonical tags in templates/CMS to use the correct absolute URL 3. Ensure consistency: canonical URL matches the URL in sitemaps, internal links, and hreflang 4. Submit affected URLs for re-inspection in GSC after deploying fix ### HREFLANG FIX: Language/Region Tag Bugs **Diagnosis:** ```bash # Check hreflang tags on page curl -s "https://example.com/page" | grep -i 'hreflang' # Check for return links (CRITICAL: hreflang must be reciprocal) # If page-en links to page-fr via hreflang, page-fr MUST link back to page-en curl -s "https://example.com/en/page" | grep -i 'hreflang' curl -s "https://example.com/fr/page" | grep -i 'hreflang' # Check hreflang in sitemap (alternative to on-page) curl -s "https://example.com/sitemap.xml" | grep -i 'hreflang' # Check x-default tag exists curl -s "https://example.com/page" | grep -i 'x-default' ``` **Common hreflang errors:** - Missing return links (non-reciprocal hreflang) - Wrong language/region codes (`en-uk` instead of `en-GB`) - Hreflang pointing to non-canonical URLs - Missing `x-default` tag - Hreflang on pages that are noindex - Hreflang URL mismatch with canonical URL **Fix approach:** 1. Map all language variants for each page 2. Ensure every hreflang annotation is reciprocal 3. Use correct ISO 639-1 (language) and ISO 3166-1 Alpha-2 (region) codes 4. Hreflang URLs must match canonical URLs exactly 5. Include `x-default` for fallback 6. Validate with hreflang testing tools or GSC International Targeting report ### SITEMAP FIX: Toxic or Broken Sitemaps **Diagnosis:** ```bash # Check robots.txt for sitemap declarations curl -s "https://example.com/robots.txt" # Fetch and validate sitemap curl -s "https://example.com/sitemap.xml" | head -50 # Check for sitemap index (multiple sitemaps) curl -s "https://example.com/sitemap_index.xml" # Count URLs in sitemap curl -s "https://example.com/sitemap.xml" | grep -c '<loc>' # Check for HTTP status of sitemap curl -sI "https://example.com/sitemap.xml" | head -5 # Check for URLs in sitemap that return non-200 curl -s "https://example.com/sitemap.xml" | grep -oP '<loc>\K[^<]+' | head -10 | while read url; do status=$(curl -sI "$url" -o /dev/null -w "%{http_code}") echo "$status $url" done ``` **Toxic sitemap indicators:** - Contains URLs that return 404, 410, or 301 - Contains noindex pages - Contains non-canonical URLs - Contains URLs blocked by robots.txt - Sitemap itself returns non-200 - Over 50,000 URLs per sitemap file (exceeds limit) - Over 50MB uncompressed (exceeds limit) - Sitemap not declared in robots.txt - Stale `<lastmod>` dates (or dates in the future) **GSC Sitemap Management:** ``` SUBMIT MISSING SITEMAPS: - Use GSC Sitemaps API/tool to submit each sitemap URL - Common pattern: submit /sitemap.xml, /sitemap-posts.xml, /sitemap-pages.xml, etc. - After submitting, monitor "Submitted" vs "Indexed" counts - If big gap between submitted and indexed, investigate excluded URLs CHECK FOR MISSING SITEMAPS: - Compare sitemap_index.xml entries vs what's submitted in GSC - Any sitemap in the index but not in GSC = submit it - Any sitemap in GSC but returning 404 = remove it ``` **Fix approach:** 1. Audit all sitemaps for toxic URLs (4xx, 5xx, noindex, non-canonical) 2. Remove or fix problematic URLs 3. Ensure all valid, indexable URLs are in sitemaps 4. Submit any missing sitemaps to GSC 5. Remove any dead sitemaps from GSC 6. Verify sitemap is declared in robots.txt 7. Regenerate sitemaps if CMS-generated ones are toxic ### ROBOTS.TXT FIX: Crawler Blocking **Diagnosis:** ```bash # Fetch current robots.txt curl -s "https://example.com/robots.txt" # Check if specific path is blocked # (manually parse Disallow rules against the target path) # Check for overly broad Disallow rules curl -s "https://example.com/robots.txt" | grep -i 'disallow' # Check for noindex in robots.txt (NOT a valid directive but some sites use it) curl -s "https://example.com/robots.txt" | grep -i 'noindex' ``` **Common robots.txt mistakes:** - `Disallow: /` blocking entire site - Blocking CSS/JS files needed for rendering - Blocking entire directories that contain indexable content - Different rules for Googlebot vs other bots causing confusion - `Noindex:` directive in robots.txt (Google ignores this since 2019) - Missing `Allow:` for specific paths within a Disallowed directory - Robots.txt returns non-200 (5xx = Google assumes full block temporarily) **Fix approach:** 1. Review all Disallow rules against intended indexable content 2. Remove overly broad blocks 3. Add specific Allow rules where needed 4. Ensure CSS and JS files are not blocked (needed for mobile-first indexing) 5. Test with GSC robots.txt Tester 6. Deploy updated robots.txt and request re-crawl of affected URLs ### NOINDEX FIX: Accidental Noindex Tags **Diagnosis:** ```bash # Check meta robots tag curl -s "https://example.com/page" | grep -i 'robots' # Check X-Robots-Tag HTTP header curl -sI "https://example.com/page" | grep -i 'x-robots' # Bulk check for noindex across pages curl -s "https://example.com/sitemap.xml" | grep -oP '<loc>\K[^<]+' | head -20 | while read url; do noindex_meta=$(curl -s "$url" | grep -i 'noindex' | head -1) noindex_header=$(curl -sI "$url" | grep -i 'noindex' | head -1) if [ -n "$noindex_meta" ] || [ -n "$noindex_header" ]; then echo "NOINDEX: $url" [ -n "$noindex_meta" ] && echo " meta: $noindex_meta" [ -n "$noindex_header" ] && echo " header: $noindex_header" fi done ``` **Common noindex causes:** - CMS "discourage search engines" setting left on from staging - Meta robots tag added during development, never removed - X-Robots-Tag header set in server config (.htaccess, nginx, Hostinger config) - Plugin or middleware injecting noindex conditionally (e.g., on pagination) - Conditional noindex based on query parameters **Fix approach:** 1. Identify source of noindex (HTML meta tag vs HTTP header vs server config) 2. Remove the noindex directive at its source 3. If CMS setting, toggle it off and verify HTML output changes 4. If server config, update .htaccess/nginx config and restart 5. After fix, request re-indexing in GSC for affected pages ### CRAWL BUDGET: Pages Crawled But Not Indexed When GSC shows many pages as "Crawled - currently not indexed": **Diagnosis:** - Are these thin/duplicate/low-quality pages? - Is the site very large (100K+ pages)? - Are important pages buried deep in site architecture? - Is crawl budget being wasted on faceted navigation, infinite scrolls, or parameter URLs? **Fix approach:** 1. Consolidate thin/duplicate content 2. Improve internal linking to important pages 3. Use `noindex` or `robots.txt` to prevent crawling of low-value pages (facets, filters, sorts) 4. Ensure important pages are within 3 clicks of homepage 5. Improve page quality signals (content depth, E-E-A-T) --- ## Phase 3: GSC MCP Integration ### Tool Discovery ``` STEP 1: Discover available GSC tools gateway_search_tools: query = "google search console" -- or -- gateway_search_tools: query = "searchconsole" STEP 2: Get schema for specific tool gateway_get_tool_schema: tool_name = "[tool_name_from_step_1]" ``` ### Common GSC Operations via MCP ``` SEARCH ANALYTICS (Traffic Data): - Query: clicks, impressions, CTR, position - Dimensions: date, page, query, country, device - Date range: specify start_date and end_date - Filters: page contains/equals, query contains/equals URL INSPECTION: - Inspect specific URL for indexing status - Get Google's canonical selection - Check mobile usability - See last crawl date SITEMAP MANAGEMENT: - List submitted sitemaps - Submit new sitemap - Delete sitemap - Check sitemap status (submitted vs indexed counts) COVERAGE REPORT (if available): - Get indexing status breakdown - See exclusion reasons and affected URLs ```