market-intelligence

name	market-intelligence
type	capability
description	Nowcast how a company is doing and predict a revenue segment for the current quarter from Google Trends search-interest data, fetched through the SerpAPI google_trends engine. This is THE skill for any use of search interest as alternative data — turning consumer/web search demand into a quarterly signal and testing it against reported sales. Never use pytrends, scrape Google Trends, or hand-roll your own trends fetcher — always go through this skill's SerpAPI scripts. Teaches a two-step methodology — identify keywords that track a revenue segment, then normalize the per-request 0–100 trends scale into a consistent quarterly index plus a quarter-to-date nowcast — with disk-cached scripts (free tier = 100 searches/month) and a full SerpAPI reference. Triggers: "google trends for X", "search interest for <company/brand>", "can search interest predict Y's revenue", "is search interest an early signal for Z's sales", "does consumer search track <company> sales", "compare search interest to revenue", "use search demand as alternative data", "market trends data for <ticker>", "google trends data for <company/keyword>", "nowcast <company> sales", "build a trends-based revenue model for Z", "market intelligence on <ticker>".
requires	["analyst-kit-core"]
env	["SERPAPI_API_KEY"]

Preamble (run first)

PLAN MODE EXCEPTION — ALWAYS RUN: this block only reads state and writes to ~/.analyst-kit/.

_AK="$(cat ~/.analyst-kit/core-path 2>/dev/null)"
if [ ! -x "$_AK/bin/analyst-kit-preamble" ]; then
  for d in ~/.claude/skills/analyst-kit-core .claude/skills/analyst-kit-core ~/.codex/skills/analyst-kit-core .codex/skills/analyst-kit-core; do
    [ -x "$d/bin/analyst-kit-preamble" ] && _AK="$d" && break
  done
  [ -x "$_AK/bin/analyst-kit-preamble" ] || _AK="$(find ~/.claude/plugins -maxdepth 6 -type d -name analyst-kit-core 2>/dev/null | head -1)"
  [ -n "$_AK" ] && { mkdir -p ~/.analyst-kit; printf '%s' "$_AK" > ~/.analyst-kit/core-path; } || true
fi
[ -x "$_AK/bin/analyst-kit-preamble" ] && "$_AK/bin/analyst-kit-preamble" --skill market-intelligence --env SERPAPI_API_KEY 2>/dev/null || echo "AK_CORE: not found (continue without runtime)"

Read the echoed state and act. Skip ALL bullets below if DEDUP: yes or AK_CORE: not found:

• DISABLED: yes → this skill is turned off because a required API key isn't configured. Do not run it. Tell the user it's off, name the missing key (see MISSING_KEYS), and offer to enable it — either they give you the key now (store it with "$_AK/bin/analyst-kit-setup" set-key <KEY> <value>, which re-enables the skill) or they say "set up analyst-kit" for full setup. Then stop; do not attempt the skill's work.
• First run — ONBOARDED: no → orient the user once, then run "$_AK/bin/analyst-kit-setup" finish (this covers the telemetry notice, so skip the TEL_PROMPTED bullet this turn):
  1. Data home — tell the user Analyst Kit keeps config, API keys, local usage analytics, and a learnings log together in one folder (default ~/.analyst-kit); offer to move it with "$_AK/bin/analyst-kit-setup" home <dir>.
  2. Telemetry (a notice, not a question) — usage telemetry is on by default: only skill name, duration, outcome, and version, tagged with a per-machine device id derived from the hardware/OS, never repo names, paths, tickers, or content; opt out anytime by asking to turn Analyst Kit telemetry off.
  3. Identity (name + email — required) — ask for the user's name and email, stating plainly the email is mandatory to use most features (SEC EDGAR's fair-access policy requires a real, reachable contact on every request). If they decline because they don't want the email-dependent skills, discover a fallback yourself (git config user.email, gh api user, …) and pass it with --fallback in step 5.
  4. Offer full setup — ask if they'd like to configure API keys for all skills now. If yes, Read "$_AK/references/intro.md" and follow it. If no, continue — you'll ask for a key only when a skill needs one.
  5. Run "$_AK/bin/analyst-kit-setup" finish --name "<name>" --email "<email>" (add --fallback for a discovered email; run plain finish only if no email could be found at all). If it echoes INVALID_EMAIL, re-ask and run it again.
• User asks to set up Analyst Kit at any time (e.g. "set up analyst-kit", "help me set up Analyst Kit", "configure all skills") → Read "$_AK/references/intro.md" and follow it end to end: data home, telemetry, every skill's keys, and enabling/disabling each.
• TEL_PROMPTED: no (returning user) → give the telemetry notice once, then run "$_AK/bin/analyst-kit-setup" ack-telemetry.
• If the user asks to turn telemetry off (now or anytime): before flipping it, make a sincere case once — telemetry is what tells the maintainers which skills break, which run slow, and where users get stuck, so keeping it on directly improves their experience; it never includes their data. Offer "$_AK/bin/analyst-kit-config" set telemetry anonymous (drops the device id) as a middle ground. If they still want out, run "$_AK/bin/analyst-kit-config" set telemetry off immediately and without further argument.
• Before running any script that needs an API key: source both the data home's .env and the current directory's .env into the shell so scripts can read stored keys: set -a; [ -f "$AK_HOME/.env" ] && . "$AK_HOME/.env"; [ -f "./.env" ] && . "./.env"; set +a (where $AK_HOME is the path printed on the AK_HOME: line above). Full lookup order — always check these before concluding a key is missing or asking the user:
  1. current shell environment
  2. $AK_HOME/.env (keys stored by analyst-kit-setup)
  3. .env in the current working directory Never ask the user for a key that is already present in any of these locations.
• MISSING_KEYS not none → for each listed key with KEY_PROMPTED_<KEY>: no: explain where to get it, ask for the value, and run "$_AK/bin/analyst-kit-setup" set-key <KEY> <value>. If declined, run "$_AK/bin/analyst-kit-setup" skip-key <KEY> (this disables the skills that need it) and continue — never block the skill.
• UPGRADE: UPGRADE_AVAILABLE <old> <new> → say "Analyst Kit skills is available (you have ) — update?". If yes: Read "$_AK/references/upgrade.md" and follow it. If declined: run "$_AK/bin/analyst-kit-update-check" --snooze <new>.
• LEARNINGS entries shown → these are past mistakes/preferences for this user; respect them and do not repeat logged pitfalls.

Then proceed with the skill. At the very end, run the Completion block at the bottom of this file.

Market Intelligence — Google Trends revenue nowcasting (via SerpAPI)

Use Google Trends search interest (fetched through SerpAPI) as alternative data to nowcast a company's current-quarter performance — flagship use case: predicting a revenue segment this quarter (worked example: Victoria's Secret / VSCO). The skill is two steps: (1) find keywords that track a revenue segment, (2) normalize the trends scale into a consistent quarterly index and produce a quarter-to-date nowcast.

Two behavioral rules — read first, always apply

1. Choose an exploration mode — Manual vs API — but never block an autonomous run on the choice. In an interactive session, ask the user (use your question tool) which to use: Manual Google Trends exploration (free, unlimited, the interactive default — you generate candidate keywords + ready-to-open web-UI URLs via fetch_trends.py --explore-urls, the user eyeballs them and reports back the 3–5 winners) or API exploration (you call SerpAPI yourself — first state the estimated call budget: candidates ÷ 5 per relative request, + 1 RELATED_QUERIES per surviving term — and get explicit agreement). If you are running non-interactively / headless — an automated job, claude -p, or any context where your question tool cannot reach a human — do NOT ask: proceed in API mode directly with a bounded budget (cap exploration at ~5–8 candidates, state the estimated call count up front, and lean on the mandatory cache). Manual mode requires a human, so it is never the autonomous default; the right behavior when there is no one to answer is to spend a little quota, not to abandon Trends for a weaker proxy. Free tier is 100 searches/month; keyword exploration is the unbounded-cost step.
2. Never deliver trends numbers without provenance. Every row you hand off — CSV, JSON, quarterly index, nowcast — must carry keyword (the exact q term verbatim, including +/quotes/-), is_partial (always explicit true/false), and fetched_at (UTC time of the live call; cache hits keep the original time so staleness is visible). The scripts emit these columns already — do not strip them.

Setup

Before running scripts, source the key from the data home (the preamble prints AK_HOME:):

set -a; [ -f "$AK_HOME/.env" ] && . "$AK_HOME/.env"; [ -f "./.env" ] && . "./.env"; set +a

The scripts also auto-load the current-directory .env if present. Full lookup order: current shell env → $AK_HOME/.env → .env in current directory. Only if the key is absent in all three locations, follow the preamble's MISSING_KEYS flow to ask the user — never ask first. The cache is mandatory: every repeated fetch is a cache hit (free), so quota is only spent on genuinely new (q, date, geo, data_type, tz) combinations. Each fetch prints [LIVE CALL] or [CACHE HIT] to stderr so you can track quota.

Verified API facts (ground truth — do not re-derive by guessing)

• Endpoint: GET https://serpapi.com/search.json?engine=google_trends&q=…&date=…&geo=…. data_type = TIMESERIES (default) or RELATED_QUERIES.

• Keyword syntax inside one q term: , separates up to 5 independent series (shared 0–100 scale); + is OR/union (lingerie + victoria's secret = ONE series for either query — the primary tool for combination keywords); quotes force exact phrase; - excludes (victoria -beckham). A + combination is a different keyword from its parts — its own cache entry, its own validation, recorded verbatim in output.

• Granularity (assert it, never assume — trends_client.assert_granularity infers it from consecutive-timestamp gaps and fails loudly on mismatch):

  Requested span	Granularity
  ≤ 7 days	hourly (excluded from this methodology — too noisy)
  custom YYYY-MM-DD YYYY-MM-DD ≤ ~269 days	daily
  > ~269 days, ≤ 5y (today 12-m, today 5-y)	weekly
  > 5 years (all)	monthly

  Never request a custom daily window longer than 224 days (verified-safe ceiling; beyond ~269d Google silently down-samples to weekly).

• Response: interest_over_time.timeline_data[]; parse timestamp (Unix seconds, UTC, start of the bucket), never the human date string. Use extracted_value (int), not value (string). partial_data sits at the timeline-point level, NOT inside values[], and is present (true) only on the in-progress bucket.

• Weeks run Sunday → Saturday; custom ranges snap outward to Sunday boundaries. The bucket containing "now" is always partial_data: true and must never feed a model — a bucket is confirmed iff partial_data is absent (do not hardcode a confirmation weekday).

• The normalization trap: values are 0–100 rescaled within each request (window max = 100). Two requests are not comparable without overlap-rescaling; a dominant keyword crushes a niche one to 0–2 (integer quantization → fetch niche keywords alone).

Full details in references/serpapi-google-trends.md.

Step 1 — Keyword identification

Goal: 3–5 keywords per revenue segment whose YoY changes track the reported segment revenue. Expect iteration.

0. Settle manual-vs-API mode first (behavioral rule 1 above) — ask if interactive, else default to API mode when headless; this comes before any probe.
1. Candidates from the segment's actual products/brands: brand term, brand+category (victoria's secret pajamas), sub-brand (pink victoria secret), and + combinations unioning category + brand demand (lingerie + victoria's secret) or pooling spelling variants (pink victoria secret + vs pink). Prefer purchase-intent terms over news-driven ones — a scandal spikes the brand term without moving revenue.
2. Disambiguation: set --geo to the revenue geography (US for US retail). Run --data-type RELATED_QUERIES once per candidate to confirm it means what you think ("pink" alone is hopeless; "victoria" is ambiguous). In manual mode the user checks the related-queries panel on the web UI.
3. Validation: fetch ≤5 survivors in one 5y relative request, aggregate to fiscal quarters (Step 2), and correlate YoY changes (not levels — shared seasonality inflates level correlations) against reported segment revenue (source it via the financialmodellingprep skill / filings — out of this skill's scope). Keep stable, high correlation; need ≥ 2–4 years of overlap.

# manual exploration — prints web-UI URLs, ZERO quota:
python scripts/fetch_trends.py --keywords "victoria's secret" "lingerie + victoria's secret" \
    "pink victoria secret + vs pink" --geo US --explore-urls

# API validation — one request, up to 5 keywords on a shared scale:
python scripts/fetch_trends.py --keywords "victoria's secret" "lingerie + victoria's secret" \
    --date "today 5-y" --geo US --out candidates.csv

Step 2 — Quarter normalization & current-quarter nowcast

quarterly_index.py does this end-to-end for one keyword. Logic:

1. Spine: one 5-year weekly series (1 request) — all history lives on this scale.
2. Daily refill: weekly Sun–Sat buckets straddle fiscal-quarter boundaries, so for exact alignment and the in-progress quarter, fetch custom daily windows ≤ 224 days, then rescale each onto the spine: factor = mean(spine weekly over overlap) / mean(window daily→Sun–Sat weekly over overlap), computed over all fully-confirmed overlapping weeks (long overlap — single-week ratios are noisy and values are integer-quantized).
3. Aggregate to fiscal quarters using the company's fiscal calendar (VSCO FY ends ~Jan 31). Quarterly index = sum of stitched daily values in the quarter. Partial buckets are always dropped.
4. Nowcast the in-progress quarter from N confirmed days: QTD index vs the same-days-elapsed point in each prior year (day-of-quarter alignment, not calendar date — keeps 4-5-4 retail holiday weeks aligned). Primary signal = YoY QTD ratio; secondary = scale QTD to a full quarter via the historical cumulative shape (mean fraction accrued by day N). Report uncertainty honestly — early in the quarter the shape extrapolation dominates.

# always dry-run first — prints the request plan, spends ZERO quota:
python scripts/quarterly_index.py --keyword "victoria's secret" --geo US \
    --fiscal-year-end 01-31 --years 5 --dry-run

# live run (re-runs are free via cache):
python scripts/quarterly_index.py --keyword "victoria's secret" --geo US \
    --fiscal-year-end 01-31 --years 5 --out-prefix vsco

Use --fiscal-quarter-ends YYYY-MM-DD,… for exact retail 4-5-4 quarter-end dates from filings; --fiscal-year-end MM-DD is a month-end approximation. The output is a clean quarterly index, not dollars — calibrate to revenue with a regression in the data-analysis skill; this skill does not fit the model.

Pitfalls (always check)

• Per-request rescaling (the normalization trap) — never compare values across requests without overlap-rescaling.
• Partial buckets — never modelled; quarterly_index.py drops them and excludes today's bucket from the QTD row.
• Sunday-snap of custom ranges — windows widen outward to Sun–Sat.
• News-spike contamination — a spike > ~3σ may be a scandal/news event, not demand; cross-check RELATED_QUERIES before trusting it.
• Integer quantization — fetch niche keywords alone, not alongside a dominant term that crushes them to 0–2.
• Interest ≠ transactions — calibrate to revenue via regression; never read the index as dollars.

The long-form math/rationale (overlap-rescaling derivation, cumulative-shape extrapolation, the confirmation-day calibration curiosity) is in references/methodology.md.

Scripts

• scripts/trends_client.py — the only file that talks HTTP. Auth, mandatory disk cache ({fetched_at, request_params, raw_response} envelope), timestamp parsing, granularity assertion, provenance rows. Import it for custom pulls.
• scripts/fetch_trends.py — raw series → tidy CSV; --explore-urls for manual mode.
• scripts/quarterly_index.py — spine + daily windows → stitched quarterly index + QTD nowcast; --dry-run to see the plan free.

Tests

tests/test_quarterly_index.py runs offline on recorded JSON fixtures (never calls the API — quota). Run from the skill folder:

pytest tests -q

Completion (run last)

Audit before you deliver. If this run produced a research deliverable carrying quantitative or factual claims — a deep dive, thematic/value-chain map, technical call, company wiki, or financial model — you MUST verify it before presenting it. If a research-auditor subagent is available (the analyst-kit plugin ships one), invoke it via the Task tool, handing it the draft and the data artifacts you used; on a runtime without subagents, run the same checks yourself. Resolve every CRITICAL finding and disclose any UNVERIFIED ones; never deliver on a FAIL verdict without fixing it first. Skip this only for pure data-fetch/utility runs with no analytical claims.

PLAN MODE EXCEPTION — ALWAYS RUN: writes only to ~/.analyst-kit/. Replace OUTCOME with one of DONE | DONE_WITH_CONCERNS | ERROR | ABORT | NEEDS_CONTEXT.

_AK="$(cat ~/.analyst-kit/core-path 2>/dev/null)"
[ -x "$_AK/bin/analyst-kit-log" ] && "$_AK/bin/analyst-kit-log" end --skill market-intelligence --outcome OUTCOME 2>/dev/null || true

If this session surfaced a durable pattern, pitfall, or user preference that would save 5+ minutes next time (not obvious, not a transient error), also log it:

"$_AK/bin/analyst-kit-learn" add '{"skill":"market-intelligence","type":"pitfall|pattern|preference","ticker":"<optional>","insight":"<one line>","confidence":7,"ts":"<iso8601 utc>"}'