theme-detector

name	theme-detector
description	Detect and analyze trending market themes across sectors. Use when user asks about current market themes, trending sectors, sector rotation, thematic investing, what themes are hot or cold, or wants to identify bullish and bearish market narratives with lifecycle analysis.

Theme Detector

Overview

This skill detects and ranks trending market themes by analyzing cross-sector momentum, volume, and breadth signals. It identifies both bullish (upward momentum) and bearish (downward pressure) themes, assesses lifecycle maturity (Emerging/Accelerating/Trending/Mature/Exhausting), and provides a confidence score combining quantitative data with narrative analysis.

3-Dimensional Scoring Model:

1. Theme Heat (0-100): Direction-neutral strength of the theme (momentum, volume, uptrend ratio, breadth)
2. Lifecycle Maturity: Stage classification (Emerging / Accelerating / Trending / Mature / Exhausting) based on duration, extremity clustering, valuation, and ETF proliferation
3. Confidence (Low / Medium / High): Reliability of the detection, combining quantitative breadth with narrative confirmation. Script output is capped at Medium; Claude's WebSearch narrative confirmation step can elevate to High.
4. Stock Leadership: Optional daily scan-hit evidence from 5D+20%, EP9M, range expansion, new highs, and high-RS stocks. When supplied, this is blended into Theme Heat v2; when absent, it lowers confidence coverage but does not force leadership to zero.
5. Theme Match Quality: Specificity of the theme match from industry participation, explicit stock-basket hits, proxy ETF confirmation, and optional offline narrative scores. This is evidence quality, not a trade recommendation.

Key Features:

• Cross-sector theme detection using FINVIZ industry data
• Direction-aware scoring (bullish and bearish themes)
• Lifecycle maturity assessment to identify crowded vs. emerging trades
• ETF proliferation scoring (more ETFs = more mature/crowded theme)
• Integration with uptrend-dashboard for 3-point evaluation
• Stock-level leadership evidence via --scan-hits
• Theme match quality via explicit stock baskets and proxy ETF confirmation
• Leader candidate evidence ranked by abnormal move/volume/range/RS metrics, with market cap shown only as a risk bucket
• Theme history and acceleration metrics via --history-file
• Dual-mode operation: FINVIZ Elite (fast) or public scraping (slower, limited)
• WebSearch-based narrative confirmation for top themes

---

When to Use This Skill

Explicit Triggers:

• "What market themes are trending right now?"
• "Which sectors are hot/cold?"
• "Detect current market themes"
• "What are the strongest bullish/bearish narratives?"
• "Is AI/clean energy/defense still a strong theme?"
• "Where is sector rotation heading?"
• "Show me thematic investing opportunities"

Implicit Triggers:

• User wants to understand broad market narrative shifts
• User is looking for thematic ETF or sector allocation ideas
• User asks about crowded trades or late-cycle themes
• User wants to know which themes are emerging vs. exhausted

When NOT to Use:

• Individual stock analysis (use us-stock-analysis instead)
• Specific sector deep-dive with chart reading (use sector-analyst instead)
• Portfolio rebalancing (use portfolio-manager instead)
• Dividend/income investing (use value-dividend-screener instead)

---

Prerequisites

Required:

• Python 3.9+ with core dependencies.

  pip install requests beautifulsoup4 lxml pandas numpy yfinance
  

Cron / mixed-Python fallback: If the active python3 is older than 3.10, or a newer Hermes venv lacks the data-science dependencies, run the detector through uv with an explicit modern interpreter and temporary dependencies instead of editing the environment mid-cron:

uv run --python 3.12 \
  --with requests --with beautifulsoup4 --with lxml \
  --with pandas --with numpy --with yfinance \
  --with finvizfinance --with PyYAML \
  python skills/theme-detector/scripts/theme_detector.py \
  --finviz-api-key "$FINVIZ_API_KEY" \
  --fmp-api-key "$FMP_API_KEY" \
  --output-dir reports/

Use this as a setup workaround, not as evidence that the detector is broken; still report FINVIZ/FMP/API-data caveats separately.

Optional API Keys:

FINVIZ Elite (recommended for full industry coverage and speed):

export FINVIZ_API_KEY=your_finviz_elite_api_key_here

FMP API (optional, for P/E ratio valuation data):

export FMP_API_KEY=your_fmp_api_key_here

Optional Python packages:

• finvizfinance - Required for FINVIZ Elite mode
• PyYAML - Required for --themes-config custom themes

Without FINVIZ Elite, the skill uses public FINVIZ scraping (limited to ~20 stocks per industry, slower rate limits).

---

Workflow

Step 1: Verify Environment

Check that API keys are configured (see Prerequisites):

# Verify FINVIZ Elite API key (optional but recommended)
echo $FINVIZ_API_KEY

# Verify FMP API key (optional)
echo $FMP_API_KEY

Step 2: Execute Theme Detection Script

Run the main detection script:

python3 skills/theme-detector/scripts/theme_detector.py \
  --output-dir reports/

Script Options:

# Full run (public FINVIZ mode, no API key required)
python3 skills/theme-detector/scripts/theme_detector.py \
  --output-dir reports/

# With FINVIZ Elite API key
python3 skills/theme-detector/scripts/theme_detector.py \
  --finviz-api-key $FINVIZ_API_KEY \
  --output-dir reports/

# With FMP API key for enhanced stock data
python3 skills/theme-detector/scripts/theme_detector.py \
  --fmp-api-key $FMP_API_KEY \
  --output-dir reports/

# Custom limits
python3 skills/theme-detector/scripts/theme_detector.py \
  --max-themes 5 \
  --max-stocks-per-theme 10 \
  --output-dir reports/

# Explicit FINVIZ mode
python3 skills/theme-detector/scripts/theme_detector.py \
  --finviz-mode public \
  --output-dir reports/

# Add Stockbee/Pradeep-style leadership evidence
python3 skills/theme-detector/scripts/theme_detector.py \
  --scan-hits data/theme_scan_hits_YYYY-MM-DD.json \
  --narrative-scores data/theme_narrative_scores_YYYY-MM-DD.json \
  --history-file reports/theme_detector_history.json \
  --as-of-date YYYY-MM-DD \
  --output-dir reports/

Scan-hit input contract: --scan-hits accepts JSON, JSONL, or CSV. Rows may be pre-labeled with scan_type / scan_types, or raw rows with fields such as symbol, return_5d, change_pct, volume, avg_volume_50d, relative_volume, true_range, atr_20, atr_expansion, close_location, industry, sector, and theme_guess. A raw row can expand into multiple hits when it satisfies multiple rules.

Initial scan rules:

• five_day_20pct: return_5d >= 20
• ep9m: volume >= 9,000,000, relative_volume >= 2.0, and change_pct >= 4
• range_expansion: change_pct >= 4, true_range / atr_20 >= 1.5 or atr_expansion >= 1.5, and close_location >= 0.75
• new_high: explicit new_high / is_new_high, or 52-week high evidence
• high_rs: rs_rating >= 90 or normalized relative_strength >= 0.90

Narrative-score input contract: --narrative-scores is an offline JSON input, not a live WebSearch call. It accepts either {"Theme Name": 82} or {"themes": {"Theme Name": {"narrative_keyword_score": 82}}}. Missing narrative input leaves narrative_keyword_score as null and reduces theme_match_coverage; it does not fail the run.

Expected Execution Time:

• FINVIZ Elite mode: ~2-3 minutes (14+ themes)
• Public FINVIZ mode: ~5-8 minutes (rate-limited scraping)

Step 3: Read and Parse Detection Results

The script generates two output files:

• theme_detector_YYYY-MM-DD_HHMMSS.json - Structured data for programmatic use
• theme_detector_YYYY-MM-DD_HHMMSS.md - Human-readable report

Read the JSON output to understand quantitative results:

# Find the latest report
ls -lt reports/theme_detector_*.json | head -1

# Read the JSON output
cat reports/theme_detector_YYYY-MM-DD_HHMMSS.json

Step 4: Perform Narrative Confirmation via WebSearch

For the top 5 themes (by Theme Heat score), execute WebSearch queries to confirm narrative strength:

Search Pattern:

"[theme name] stocks market [current month] [current year]"
"[theme name] sector momentum [current month] [current year]"

Evaluate narrative signals:

• Strong narrative: Multiple major outlets covering the theme, analyst upgrades, policy catalysts
• Moderate narrative: Some coverage, mixed sentiment, no clear catalyst
• Weak narrative: Little coverage, or predominantly contrarian/skeptical tone

Update Confidence levels based on findings:

• Quantitative High + Narrative Strong = High confidence
• Quantitative High + Narrative Weak = Medium confidence (possible momentum divergence)
• Quantitative Low + Narrative Strong = Medium confidence (narrative may lead price)
• Quantitative Low + Narrative Weak = Low confidence

Step 5: Analyze Results and Provide Recommendations

Cross-reference detection results with knowledge bases:

Reference Documents to Consult:

1. references/cross_sector_themes.md - Theme definitions and constituent industries
2. references/thematic_etf_catalog.md - ETF exposure options by theme
3. references/theme_detection_methodology.md - Scoring model details
4. references/finviz_industry_codes.md - Industry classification reference

Analysis Framework:

For Hot Bullish Themes (Heat >= 70, Direction = Bullish):

• Identify lifecycle stage (Emerging = opportunity, Mature/Exhausting = caution)
• List top-performing industries within the theme
• Recommend proxy ETFs for exposure
• Flag if ETF proliferation is high (crowded trade warning)

For Hot Bearish Themes (Heat >= 70, Direction = Bearish):

• Identify industries under pressure
• Assess if bearish momentum is accelerating or decelerating
• Recommend hedging strategies or sectors to avoid
• Note potential mean-reversion opportunities if lifecycle is Mature/Exhausting

For Emerging Themes (Heat 40-69, Lifecycle = Emerging):

• These may represent early rotation signals
• Recommend monitoring with watchlist
• Identify catalyst events that could accelerate the theme

For Exhausted Themes (Heat >= 60, Lifecycle = Exhausting):

• Warn about crowded trade risk
• High ETF count confirms excessive retail participation
• Consider contrarian positioning or reducing exposure

Step 6: Generate Final Report

Present the final report to the user using the report template structure:

# Theme Detection Report
**Date:** YYYY-MM-DD
**Mode:** FINVIZ Elite / Public
**Themes Analyzed:** N
**Data Quality:** [note any limitations]

## Theme Dashboard
[Top themes table with Heat, Direction, Lifecycle, Confidence]

## What Changed Today
[Newly emerging themes, largest heat acceleration, new EP9M clusters, fading themes]

## Leadership Evidence
[5D+20%, EP9M, range expansion, new highs, high-RS counts and leader symbols]

## Bullish Themes Detail
[Detailed analysis of bullish themes sorted by Heat]

## Bearish Themes Detail
[Detailed analysis of bearish themes sorted by Heat]

## All Themes Summary
[Complete theme ranking table]

## Industry Rankings
[Top performing and worst performing industries]

## Sector Uptrend Ratios
[Sector-level aggregation if uptrend data available]

## Methodology Notes
[Brief explanation of scoring model]

Save the report to reports/ directory.

---

Output

The skill generates two output files in the reports/ directory:

JSON Output (theme_detector_YYYY-MM-DD_HHMMSS.json):

{
  "report_type": "theme_detector",
  "generated_at": "2026-04-18 10:30:00",
  "metadata": {
    "generated_at": "2026-04-18 10:30:00",
    "data_mode": "full",
    "finviz_mode": "elite",
    "fmp_available": true,
    "max_themes": 14,
    "max_stocks_per_theme": 5,
    "data_sources": {
      "finviz_industries": 152,
      "yfinance_stocks": 68,
      "etf_volume": 24
    }
  },
  "summary": {
    "total_themes": 14,
    "bullish_count": 8,
    "bearish_count": 6,
    "top_bullish": "AI & Machine Learning",
    "top_bearish": "Regional Banks"
  },
  "themes": {
    "all": [
      {
        "name": "AI & Machine Learning",
        "direction": "bullish",
        "heat": 85.3,
        "maturity": 42.1,
        "stage": "Accelerating",
        "confidence": "Medium",
        "heat_label": "Hot",
        "industries": ["Software - Infrastructure", "Semiconductors"],
        "representative_stocks": ["NVDA", "MSFT"],
        "stock_details": [{"symbol": "NVDA"}, {"symbol": "MSFT"}],
        "proxy_etfs": ["BOTZ", "ROBO"],
        "theme_match_score": 78.4,
        "theme_match_components": {
          "industry_match_score": 84.2,
          "static_stock_hit_score": 80.0,
          "proxy_etf_momentum_score": 70.0,
          "narrative_keyword_score": null
        },
        "leader_candidates": [
          {
            "symbol": "NVDA",
            "leader_score": 91.2,
            "scan_types": ["ep9m", "range_expansion"],
            "risk_bucket": "mega"
          }
        ],
        "theme_origin": "seed"
      }
    ],
    "bullish": [...],
    "bearish": [...],
    "match_ranked": [...]
  },
  "industry_rankings": {
    "top": [...],
    "bottom": [...]
  },
  "sector_uptrend": {...},
  "data_quality": {...}
}

Markdown Report (theme_detector_YYYY-MM-DD_HHMMSS.md):

• Theme Dashboard with sortable rankings
• Bullish/Bearish theme detail sections
• Industry performance rankings
• Sector uptrend ratio summary
• Methodology notes

Key Output Fields (per theme):

Field	Description
heat	0-100 direction-neutral theme strength
direction	"bullish" (LEAD) or "bearish" (LAG)
stage	Emerging / Accelerating / Trending / Mature / Exhausting
confidence	Low / Medium / High (script caps at Medium; WebSearch can elevate)
representative_stocks	Top ticker symbols for the theme
stock_details	Optional stock metric objects for the selected representatives
proxy_etfs	Thematic ETF tickers (length = ETF count; higher = more crowded)
theme_match_score	0-100 evidence-quality score from industries, stock basket hits, proxy ETF confirmation, and optional narrative input
theme_match_components	Inspectable sub-scores explaining the theme match
leader_candidates	Evidence-ranked symbols for the theme; not entry/stop/invalidation guidance
fresh_leadership_symbols	Current-run symbols with EP9M, range expansion, or new-high evidence
extended_symbols	Current-run symbols with 5D+20% evidence; used as overextension evidence only
theme_origin	"seed" (from YAML config) or "discovered" (auto-clustered)

---

Resources

Scripts Directory (scripts/)

Main Scripts:

• theme_detector.py - Main orchestrator script

  • Coordinates industry data collection, theme classification, and scoring
  • Generates JSON + Markdown output
  • Usage: python3 theme_detector.py [options]

• theme_classifier.py - Maps industries to cross-sector themes

  • Reads theme definitions from cross_sector_themes.md
  • Calculates theme-level aggregated scores
  • Determines direction (bullish/bearish) from constituent industries
  • Display mapping: "bullish" → "LEAD", "bearish" → "LAG" (see report_generator.py::_direction_label())

• finviz_industry_scanner.py - FINVIZ industry data collection

  • Elite mode: CSV export with full stock data per industry
  • Public mode: Web scraping with rate limiting
  • Extracts: performance, volume, change%, avg volume, market cap

• calculators/lifecycle_calculator.py - Lifecycle maturity assessment

  • Duration scoring, extremity clustering, valuation analysis
  • ETF proliferation scoring from thematic_etf_catalog.md
  • Stage classification: Emerging / Accelerating / Trending / Mature / Exhausting

• report_generator.py - Report output generation

  • Markdown report from template
  • JSON structured output
  • Theme dashboard formatting

References Directory (references/)

Knowledge Bases:

• cross_sector_themes.md - Theme definitions with industries, ETFs, stocks, and matching criteria
• thematic_etf_catalog.md - Comprehensive thematic ETF catalog with counts per theme
• finviz_industry_codes.md - Complete FINVIZ industry-to-filter-code mapping
• theme_detection_methodology.md - Technical documentation of the 3D scoring model

Assets Directory (assets/)

• report_template.md - Markdown template for report generation with placeholder format

---

Important Notes

FINVIZ Mode Differences

Feature	Elite Mode	Public Mode
Industry coverage	All ~145 industries	All ~145 industries
Stocks per industry	Full universe	~20 stocks (page 1)
Rate limiting	0.5s between requests	2.0s between requests
Data freshness	Real-time	15-min delayed
API key required	Yes ($39.50/mo)	No
Execution time	~2-3 minutes	~5-8 minutes

Direction Detection Logic

Theme direction is determined by majority vote of constituent industries' relative rank:

1. Industry ranking: All ~145 industries are ranked by multi-timeframe momentum score
2. Rank-based direction: Industries in the top half of the ranked list are classified as "bullish"; bottom half as "bearish"
3. Theme majority vote: _majority_direction() counts bullish vs. bearish industries within each theme; the majority wins

Display mapping: "bullish" → LEAD, "bearish" → LAG (see report_generator.py::_direction_label())

A LEAD theme indicates relative outperformance of its constituent industries. A LAG theme may still have positive absolute returns — it indicates relative underperformance, not a short signal.

Known Limitations

1. Survivorship bias: Only analyzes currently listed stocks and ETFs
2. Lag: FINVIZ data may lag intraday moves by 15 minutes (public mode)
3. Theme boundaries: Some stocks fit multiple themes; classification uses primary industry
4. ETF proliferation: Catalog is static and may not capture very new ETFs
5. Narrative scoring: WebSearch-based and inherently subjective
6. Public mode limitation: ~20 stocks per industry may miss small-cap signals

Disclaimer

This analysis is for educational and informational purposes only.

• Not investment advice
• Past thematic trends do not guarantee future performance
• Theme detection identifies momentum, not fundamental value
• Conduct your own research before making investment decisions

---

Version: 1.0 Last Updated: 2026-02-16 API Requirements: FINVIZ Elite (recommended) or public mode (free); FMP API optional Execution Time: ~2-8 minutes depending on mode Output Formats: JSON + Markdown Themes Covered: 14+ cross-sector themes