apify-ai-search-visibility-tracker

name	apify-ai-search-visibility-tracker
description	Track whether a brand and its competitors get cited or mentioned across Google AI Overviews, Google AI Mode, ChatGPT Search, Perplexity, Microsoft Copilot, and Google Gemini for a defined set of prompts, on a recurring schedule. Use when user asks to track AI visibility, monitor brand mentions in AI search, track ChatGPT citations, do AI search SEO tracking, GEO tracking (Generative Engine Optimization), AEO tracking (Answer Engine Optimization), monitor Perplexity citations, track AI Overviews mentions, see if their brand shows up in AI search, discover which prompts competitors rank for in AI search, find citation opportunities, or audit a website for AI visibility readiness.
author	Daniela Ryplová
author_url	https://github.com/danielarypl
metadata	{"keywords":"ai-search, geo, aeo, generative-engine-optimization, answer-engine-optimization, brand-visibility, citations, ai-overviews, chatgpt-search, perplexity, copilot, gemini, google-search-scraper, monitoring, scheduling"}

AI Search Visibility Tracker

Four workflows covering the full AI visibility lifecycle: discover which prompts matter → find citation opportunities → audit your site → track over time.

All workflows use apify/google-search-scraper for AI search. Workflow C also uses apify/website-content-crawler.

Recommended flow: Run Workflow A to discover prompts → Workflow B to find citation opportunities → Workflow C to audit your site → Workflow D to track everything on a schedule.

---

Workflow A — Competitor Prompt Discovery

Goal: Find which queries surface a competitor in AI search answers, so you know which prompts are worth monitoring.

Inputs to collect

#	Input	Notes
1	Competitor domain(s)	e.g. brightdata.com, scraperapi.com
2	Seed topic keywords	e.g. "web scraping", "data extraction API"
3	AI sources	Default: all six (AI Overviews, AI Mode, ChatGPT, Perplexity, Copilot, Gemini)

Workflow

1. Generate 15–30 candidate queries from seed keywords using these templates:

   • best [topic], [topic] tools, how to [topic], [topic] for [use case]
   • [topic] vs [competitor brand], [competitor brand] alternative
   • [topic] API, [topic] pricing, [topic] tutorial

2. Run apify/google-search-scraper for each candidate query. For each result, extract:

   • aiOverview.sources[], aiMode.sources[], chatGptAnswer.sources[], perplexityAnswer.sources[], copilotAnswer.sources[], geminiAnswer.sources[]
   • Also check answer_text / aiOverview.text for competitor brand name mentions (word-boundary match: \bBrand\b)

3. For each (query, source) pair where the competitor domain or brand appears: record a hit.

4. Output a prompt-major table sorted by total hit count descending:

| Query | ChatGPT | Perplexity | AI Overviews | AI Mode | Copilot | Gemini | Total |
|-------|---------|------------|--------------|---------|---------|--------|-------|
| "best web scraping API" | ✓ | ✓ | — | ✓ | — | ✓ | 4 |
| "how to scrape Google" | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 6 |

5. Deliver the top-N queries (default 10) as a ready-to-paste list for Workflow D's config.json prompts.

---

Workflow B — Citation Opportunity Finder

Goal: For a target topic, identify which domains and content types AI engines most often cite — revealing where to publish or pitch content.

Inputs to collect

#	Input	Notes
1	Target topic / industry	e.g. "web scraping", "ecommerce automation"
2	Seed queries	5–20 queries covering the topic space
3	AI sources	Default: all six
4	Deep-crawl top cited?	Optional: crawl top-3 cited pages with website-content-crawler for structure patterns

Workflow

1. Run apify/google-search-scraper for each seed query across selected AI sources.

2. Collect every URL from sources[] across all results. Normalise to registrable domain (blog.example.com → example.com).

3. Aggregate:

   • By domain: count citations, list which AI sources cite it, list which queries triggered it
   • By content type: infer from URL path patterns (docs → /docs/, /reference/; blog → /blog/; news → known news domains)

4. Rank by total citation count. Output:

Top-cited domains for "web scraping" (42 queries × 6 sources):
| Domain | Citations | AI Sources | Inferred type |
|--------|-----------|------------|--------------|
| docs.apify.com | 38 | ChatGPT, Perplexity, AI Mode | Documentation |
| scraperapi.com/blog | 21 | AI Overviews, Gemini | Long-form blog |

5. If deep-crawl enabled: run apify/website-content-crawler on the top-3 cited URLs per domain. From the markdown output, extract:

   • First heading that directly answers the query
   • Presence of code blocks in first 500 words
   • Word count
   • Whether an H2/H3 contains the exact query phrase

6. Summarise patterns: "AI engines in this topic prefer [long-form docs / short direct-answer posts]. Typical cited page: [word count range], [has/lacks direct-answer H2], [has/lacks code example above the fold]."

---

Workflow C — GEO Website Audit

Goal: Check whether a specific website's content is structured for AI citation; compare it against what AI engines actually cite for your target prompts.

Inputs to collect

#	Input	Notes
1	Your website URL	e.g. https://apify.com
2	Target prompts	Use Workflow A output, or supply 5–10 directly
3	AI sources	Default: all six

Workflow

1. Run apify/google-search-scraper for each target prompt. For each (prompt × source) record whether your registrable domain appears in sources[].

2. For prompts where your domain is not cited: identify the top-cited competitor URL for that prompt.

3. Run apify/website-content-crawler on:

   • Your most relevant page(s) for each un-cited prompt
   • The top-cited competitor page for each un-cited prompt

4. For each un-cited prompt, produce a gap card:

Prompt: "how to scrape Google search results"
Your page: apify.com/blog/scraping-google  →  NOT cited on ChatGPT, Perplexity, AI Mode
Top-cited: docs.brightdata.com/scraping/google (cited 5/6 sources)

Structural gaps:
  ✗ Your page: answer buried after 900 words, no direct-answer H2
  ✓ Competitor: H2 "How to scrape Google in 3 steps" at word 120 + code block at word 180

Recommended actions (priority order):
  1. Add H2 that mirrors the query phrase within first 300 words
  2. Move code example above the fold
  3. Add a "Quick answer" summary box at the top

5. Deliver: per-prompt gap cards + a consolidated action table ranked by expected impact.

---

Workflow D — Recurring Visibility Tracker

Goal: Snapshot brand citations and mentions across all six AI surfaces on a recurring schedule and track changes over time.

Prerequisites

(No need to check upfront)

• APIFY_TOKEN saved in a .env file next to config.json (the runner auto-loads it).
• Python 3.9+ on PATH. pip3 install requests (only third-party dependency); pip3 install tldextract recommended for accurate registrable-domain matching on multi-part TLDs.
• For automated daily runs: macOS / Linux with launchd or cron available (the installer handles both). Windows users get printed schtasks instructions.

Steps

Copy this checklist and track progress:

Task Progress:
- [ ] Step 1: Load or collect the seven required inputs
- [ ] Step 2: Confirm AI sources and cadence
- [ ] Step 3: Write config.json + .env, then install the OS schedule
- [ ] Step 4: Run a snapshot now so the user sees the first report
- [ ] Step 5: Deliver the history report (diff vs. all prior runs)

Step 1: Load or Collect the Seven Required Inputs

If config.json exists in the user's working directory, load it and skip to Step 4 unless the user asks to reconfigure. On first run, ask all seven anchors before any Actor call:

#	Input	Why it matters
1	Brand URL	Primary domain. Drives registrable-domain citation matching (blog.apify.com -> apify.com).
2	Brand name(s)	Surface forms for text-mention matching (e.g., Apify, apify.com, @apify). URL-only matching misses mentions without links.
3	Competitor brands	Ask explicitly: "Which competitors do you want tracked alongside your brand?" Accept name + domain pairs. Zero is allowed; the question must still be asked on first run.
4	Prompts to monitor	One or more search queries. Each runs through every enabled AI source. If you don't know which prompts to use yet, run Workflow A first — it discovers competitor-visible prompts you can paste here.
5	Cadence	daily / weekly / monthly. Drives the schedule entry that install_cron.sh writes.
6	Which AI sources	Present the six (AI Overviews, AI Mode, ChatGPT, Perplexity, Copilot, Gemini), all enabled by default. Each adds per-result cost -- current pricing on the Actor page (https://apify.com/apify/google-search-scraper).
7	Apify Dataset name	The named dataset to append to. If absent, created on first run; the name is recorded in config.json.

After those seven, ask optional follow-ups: countryCode, languageCode, location (UULE), preferred run hour (default 09:00 local).

Then one verbosity question -- save as config.json:include_full_answers:

• on_demand (default): report shows short quoted snippets around each surface-form match. Full LLM answers live in the named KV store; user can ask later.
• always: report embeds the full LLM answer verbatim whenever any entity is mentioned. Useful for one prompt; gets unwieldy at 5+ prompts.

Step 2: Confirm AI Sources and Cadence

Echo back the user's seven choices in a single paragraph for confirmation. If the user toggles sources, update the in-memory config before writing.

Step 3: Write config.json + .env, Then Install the OS Schedule

Create the working directory layout next to where the user wants reports to land:

working-dir/
  config.json     # copied from the skill's config.example.json, edited with collected values
  .env            # APIFY_TOKEN=apify_api_xxx   (chmod 600)

cp ${CLAUDE_PLUGIN_ROOT}/reference/scripts/config.example.json ./config.json
# then edit with the collected values, save

echo 'APIFY_TOKEN=your_token_here' > ./.env
chmod 600 ./.env

Then install the OS schedule:

bash ${CLAUDE_PLUGIN_ROOT}/reference/scripts/install_cron.sh --cadence daily --hour 9

Cron expression mapping:

Cadence	Cron expression	When
daily	0 H * * *	every day at H:00 local
weekly	0 H * * 1	every Monday at H:00
monthly	0 H 1 * *	the 1st of every month at H:00

Step 4: Run a Snapshot Now

macOS:

launchctl kickstart "gui/$(id -u)/com.apify.ai-visibility-tracker"
tail -f ~/Library/Logs/ai-visibility-tracker.log

Linux / generic:

python3 ${CLAUDE_PLUGIN_ROOT}/reference/scripts/run_snapshot.py --config ./config.json

Both paths:

1. Call apify/google-search-scraper with the configured prompts + AI-source toggles.
2. Parse each (prompt x source) cell for citations and brand/competitor mentions.
3. Append rows to the named Apify Dataset (schema in reference/output-schema.md).
4. Write the raw item to the named KV store (long-term archive).
5. Compute the history vs. all prior runs.
6. Write reports/snapshot-<ISO-date>.md next to config.json.

Step 5: Deliver the History Report

Open reports/snapshot-<ISO-date>.md and surface the top findings in chat. Lead with:

• First-ever citations / mentions today -- entity x source combinations crossing the threshold for the first time.
• Drops -- entity was cited in the latest prior run but isn't today.
• For every cited entity, the exact matched URL(s).

Output format

Snapshot summaries are entity-major: one Markdown table per tracked entity (brand, then competitors), one row per AI source, columns Source | Cited | Mentioned | SoV% | Matched URLs | History.

---

Actors

Actor	Used in	Key input fields
apify/google-search-scraper	A, B, C, D	queries[], aiOverview, chatGptSearch, perplexitySearch, googleAiMode, bingCopilotSearch, googleGeminiSearch
apify/website-content-crawler	B (optional), C	startUrls[], maxCrawlPages, outputMarkdown: true

Pricing changes; check the pricing tab before quoting numbers. Disable unused AI sources to reduce cost.

Telemetry. Workflow D runs through reference/scripts/run_snapshot.py, which already sets User-Agent: apify-awesome-skills/ai-search-visibility-tracker-... on every Apify API call. When calling these Actors ad-hoc in Workflows A-C, pass the matching flags so usage is attributed: --user-agent apify-awesome-skills/apify-ai-search-visibility-tracker and --json (use --format json for apify datasets get-items), and append 2>/dev/null. Example:

apify actors call apify/google-search-scraper \
  --user-agent apify-awesome-skills/apify-ai-search-visibility-tracker \
  --json 2>/dev/null

Quality Rules

• Non-interactive. No stdin reads in run_snapshot.py -- launchd / cron has no stdin.
• Word-boundary brand matching (\bbrand\b, case-insensitive). See reference/citation-matching.md.
• Registrable-domain citation matching (blog.apify.com counts as apify.com). See reference/citation-matching.md.
• Never skip a row. If an AI source returns nothing, write a row with cited: false, mentioned: false, answer_text: "[no answer returned]".
• Every row carries the Apify run ID so any finding can be reverified.

Error Handling

APIFY_TOKEN not found -- Tell the user to put it in .env next to config.json (echo 'APIFY_TOKEN=...' > .env && chmod 600 .env). Token at https://console.apify.com/account/integrations. config.json not found -- Run Step 3 first to create it from the template. Dataset name not set -- Ask the user for a name; the runner will create the dataset on first append. Actor run FAILED -- Print the console link from the runner output and ask the user to inspect it. AI source returned no answer -- The row is still written with [no answer returned]. Not an error. website-content-crawler returns no markdown -- Page may be JS-heavy; try with useBrowserCrawler: true. Schedule not firing -- See reference/scheduling.md troubleshooting section. No previous run to diff against -- First run only. The report renders the snapshot without a history section.