cti-expert

# CTI Expert Cyber threat intelligence and open-source intelligence skill. Turns Claude into a trained CTI/OSINT analyst. Generates precision search queries, interprets public data, builds case timelines, and delivers structured intelligence products — no API keys, no paid subscriptions. > **Runs anywhere.** Works in **Claude Code** (Desktop & CLI) and in **OpenAI Codex / ChatGPT** and other `AGENTS.md`-aware agents — see [`AGENTS.md`](AGENTS.md) for the cross-agent runtime contract. Throughout this file, **`$SKILL_DIR`** = the directory containing this `SKILL.md` (Claude Code: `~/.claude/skills/cti-expert`; Codex/manual clone: the repo you are working in). Resolve it by locating `SKILL.md` — never hard-assume `~/.claude`. Detect the OS once (Windows/macOS/Linux) and prefer **uv** for all Python — see §13 Tool Auto-Install Policy. Collection method: `agent-browser` when available (JavaScript-heavy sites, infinite-scroll, screenshot evidence), with automatic fallback to web search / web fetch / direct URL fetch. Tool limitations are logged as collection gaps — never as case blockers. --- ## 1. Quick Start ```bash # Full autonomous case — runs every applicable technique /case target.com # Guided flow for first-time investigators /flow person # Summary of what's been found so far /brief ``` Append `--yolo` to any command to skip all interactive prompts and confirmations. The analyst makes every decision autonomously. --- ## 2. AEAD Case Lifecycle Every investigation follows four phases: | Phase | What Happens | |-------|-------------| | **Acquire** | Collect raw data — `/sweep`, `/query`, `/username`, `/phone`, `/email-deep`, `/breach-deep`, `/subdomain`, `/webpivot` + `/icp` (domain/URL targets), `/dork-sweep` · `/docleak` · `/github-osint`/`/secrets`, `/cn-corp` · `/iban` · `/hash-id` on discovery | | **Enrich** | **Recursive pivot loop** — the [pivot orchestration engine](engine/pivot-orchestration.md) treats every discovered identifier as a new seed and expands the graph hop-by-hop (`/branch`, `/crossref`, `/link-subjects`, `/signatures`) **automatically until the frontier is exhausted**, no approval prompts (`autonomy=auto`). Each discovered identifier auto-fires its leak/breach/OSINT/dork legs — email→`/breach-deep`+`/intelx` (breach dumps·**infostealer logs**·pastes·darknet), username→`/username`+socials, name→`/dork-sweep`+`/docleak`, apex→`/intelx --phonebook`+`/secrets`+`/github-osint` — see §"Leak / breach / infostealer auto-fire" + the Dork/GitHub auto-fire matrices. Acquire↔Enrich iterate, not run once. | | **Assess** | Score and verify — `/exposure`, `/threat-model`, `/validate`, `/coverage`, `/verify-finding`. Judgments carry **likelihood terms**, coverage gets the **5W1H pass**, attributions get an **ACH matrix** ([`handbook/analytic-standards.md`](handbook/analytic-standards.md)). **If the case has not converged** (frontier still open after the pivot loop + deterministic pipeline) and posture is active, `/case` **auto-escalates to the `/harness` deepening loop** — keyless-first (the CLI's own model), egress hard-gated on hostile infra; `--no-harness` opts out | | **Deliver** | Package output — `/report`, `/brief`, `/render`, `/workspace save` — **first ASKS whether to import more evidence from manual investigation** (merged into the report JSON before anything is built), **always auto-saves the base data bundle (.md + .json + .csv + IOC bundle: .stix.json/.txt/.csv/.jsonl), then ASKS which presentation report to render — (a) PDF · (b) DOCX · (c) HTML · (d) all** (both prompts skipped under `--yolo`/guided-auto, which default to HTML). When `CHONGLUADAO_API_KEY` is set, the IOC bundle also attaches CLD's **STIX + MISP indicator feed** as companion artifacts (`cld_api.py feed stix2\|misp --raw` → loadable bundle, not merged into the case graph). **Deep-layer persist (automatic, ZERO extra egress):** when `/backend` is live, `/case` **reuses the pivots it already collected** — never re-fetches — to persist the versioned case at `$SKILL_DIR/intel_engine/cases/<CASE-ID>/` and correlate it cross-case; see the auto-chain note below. See [`connectors/chongluadao-api.md`](connectors/chongluadao-api.md) | Run `/progress` at any point to see which phase you're in and what's pending. > **`/case` and web-infra pivoting.** For a **domain or URL** target, `/case` includes > web-infrastructure pivoting (`/webpivot`) in the Acquire phase. It runs **keyless by default** > (crt.sh + passive DNS + anonymous urlscan) and **upgrades automatically when premium keys are > set** via `/apikeys` (Shodan/Censys/FOFA/Hunter.how/DNSLytics/SecurityTrails/urlscan-PRO/WhoisXML). With keys > the pipeline also: reads the **urlscan Pro hostname lifecycle** (pre-registration NS/A eras on the > timeline, verdict rows in Appendix B), runs the **MO-neighbour pivot** on the estate's non-CDN origin > (co-tenants WHOIS-verified; only a registrant join-key ever seeds, same-MO personas render as a > rung-10 *Related personas* table — `--mask-personas` to aggregate), **measures entitlement** > (`meta.capability.plans`, per-case `capability_plans.json`; Censys' search is its own probe), and > fires SecurityTrails DNS-history + DSL reverse-WHOIS, DNSLytics reverse-IP (co-tenancy-filtered), > a once-per-case Censys cert search, Shodan cert/JARM search and IntelX (loop: `--full` only). Every > metered leg is `--free-only`/`no_spend`-gated; IntelX selectors, WhoisXML/SecurityTrails reverse-WHOIS terms, > MO-neighbour origins, DNSLytics reverse-IP and the Censys cert search are bought once per CASE (on-disk memo), > while per-host legs (urlscan lifecycle, SecurityTrails subdomains/history, Shodan search) stay per host under > per-case caps. Because > `/webpivot` can fetch the target directly, for hostile infrastructure it prefers passive capture > (urlscan/Wayback) — see [`techniques/web-pivot.md`](techniques/web-pivot.md). It is **not** run for > username/phone/person targets. > > **⭐ ChongLuaDao is the first-party premium upgrade.** With `CHONGLUADAO_API_KEY` set > (`/apikeys set chongluadao <KEY>`), Acquire/Enrich fold CLD's own datasets into `/scam-check`, > `/threat-check`, `/phone`, `/breach-deep`, `/email-deep`, `/vuln-check` and `/impersonate`, and > `/cld <target>` is the direct entry point. Your client connects **only to ChongLuaDao, never > to the target** (provable from `scripts/cld/cld_api.py`); for URL/AI/IP checks CLD does any > target fetch server-side — the safe first-touch verdict on a live scam funnel before a direct > pivot. Full catalog + AEAD placement: [`connectors/chongluadao-api.md`](connectors/chongluadao-api.md). > > **Archive IOC harvest runs by default too.** For domain/URL targets the Acquire phase also runs > `wayback_harvest.py <domain> --indicators` (add `--urlscan` when `URLSCAN_API_KEY` is set), > harvesting **emails, phones, crypto wallets, tracking/verification IDs, SaaS-operator IDs, and > socials from the *entire* Wayback history** — not just the live page — with first-seen/last-seen > per selector. It writes case-schema `indicators[]` to `<case>/raw/harvest.indicators.json`, which > merge into the case and flow into the **auto-saved IOC bundle** at Deliver. This is the step that > recovers selectors a network later scrubbed — across the whole snapshot corpus, not just the live page. > Passive by construction — only web.archive.org (+ urlscan.io if keyed), never the target. > > **The five v2.6 commands are in the pipeline too — no flags.** `/icp` runs for every > domain/URL/org target (and an IP's resolved hostname); `/cn-corp`, `/iban` and `/hash-id` > fire the moment a company name/USCC, payment detail, or hash appears — and all three feed > their yields **back into the recursive pivot loop** as new seeds, so an ICP licence serial or > a reused bank account expands the graph like any other node. `/redact` is the exception: it > is **opt-in** (`--redact`), because a redacted report is a weaker artifact and that should > always be a deliberate choice. Full trigger table: §Technique Activation Matrix. > Narrow with `--no-cn`. > **Two layers, one skill: broad collector → deep pipeline.** cti-expert is the **broad > collector** — the wide net of Acquire/Enrich commands (`/webpivot`, `/sweep`, `/subdomain`, > `/icp`, `/username`, `/email-deep`, `/breach-deep`, …) that pull artifacts from anywhere. The > **`intel_engine` engine is now vendored in-repo under `intel_engine/`** (`intel_engine/harness/`, > `intel_engine/tools/`, `intel_engine/WebPivot/`, `intel_engine/IntelGraph|IntelReport|BinaryPivot|IntelAnalysis/`) > and supplies the **pipeline chains + deeper pivoting logic**: a persistent knowledge base (`intel_engine/knowledge/`), versioned cases > (`cases/`), cross-case correlation, calibrated assessment, and rendering. > > **The chain (automatic for `/case`, and it NEVER re-fetches):** broad collection (cti-expert) > already ran `pivot_extract` per host during Acquire. When `/backend` resolves to Tier 1/2 **and** > the run produced ≥1 host seed (domain/URL/IP), `/case` hands what it already collected to the > deterministic pipeline in **reuse mode** (`--no-collect`) — one command, a complete case dir, and > nothing touches the target again: > > 1. Write each host's already-collected pivot JSON to `$SKILL_DIR/intel_engine/cases/<CASE-ID>/raw/<host>.json` > and the host list to an absolute `<CASE-ID>-seeds.txt`, both **anchored at the engine root** > (`ROOT=$SKILL_DIR/intel_engine`) — never a CWD-relative path. > 2. `intel.py pipeline open <CASE-ID> <abs-seeds.txt> --no-collect` — `--no-collect` skips the live > fetch and runs the *rest of the existing pipeline* over the raw you just wrote: ingest → > prior-overlap (`/recall`) → risk (`/risk`) → shared cluster seeds → `clusters.json` → > `case_graph.json` → ICD-203 `assessment.md`. Every step is a KB/local read: zero egress, zero > metered calls. > > **The persisted case lands at `$SKILL_DIR/intel_engine/cases/<CASE-ID>/` — `raw/`, `shared.txt`, > `clusters.json`, `case_graph.json`, `assessment.md` — NOT the current working directory**, and it > is a COMPLETE case that `intel.py pipeline status <CASE-ID>` accepts (not the partial dir a hand-rolled > op sequence would leave). `<CASE-ID>` is the same id as the report filenames (mint > `CASE-YYYYMMDD-NN` when none is given). Its cluster/risk/assessment fold back into the auto-saved > report. Tier 3 (stateless), or a person/username/phone target with no host seeds → the handoff is > **skipped silently** and the tier noted; broad collection + report are unaffected. > > **Collecting `pipeline open` (no `--no-collect`) stays MANUAL — `/case` never runs it.** > `intel.py pipeline open` without the flag re-fetches every seed directly (`collect_many` on > `https://<host>`; the egress gate at `collect_core.py:169` only fires when `hostile` is set, which > the `open` path never sets), so a blind auto-`open` would be a **second live round** against infra > `/case` may have just classified hostile/no-touch — which is why the automatic handoff uses > `--no-collect`. Run collecting mode by hand only for a **fresh** case with no prior collection, > after setting the egress posture (`/scope`, `/cti-proxy`, or `--passive`). > > **`/harness` deepening AUTO-ESCALATES on non-convergence** (`--no-harness` opts out). After the > `--no-collect` pipeline, if the case has **not converged** — `intel.py convergence <case>` > reports status ≠ `converged` (`cold` = no free leads left = exhausted), or `intel.py frontier > <case>` still lists open leads > **and** posture is active (not `--passive`, infra not classified hostile/no-touch) — `/case` runs > the harness Collect→Correlate→Assess loop to close the gap. Egress stays safe **by construction**: > the harness's own `audit.py` PreToolUse gate turns `hostile=True` into a **hard denial** of > outbound collection ([`harness/README.md`](intel_engine/harness/README.md)), so an escalation can > never re-touch no-touch infra — on hostile infra it deepens correlation/assessment only. > **Model — the CLI's own agent:** run interactively in Claude Code, the IntelHarness skill > front-end drives the same pipeline on your subscription with **no separate LLM key**; it falls > back to `HARNESS_BACKEND=local` (Ollama/vLLM/LM Studio, keyless) or an API key only for unattended > SDK runs. No reasoning backend reachable **and** non-interactive → the escalation is **skipped and > noted as a collection gap**, never a blocker. A converged case, `--passive`, or hostile-only infra > → no escalation and the deterministic result stands. > > **Self-contained & self-resolving.** `/backend` resolves to **SELF** (in-repo) — no external > setup. The bundled installer (`scripts/install.{sh,ps1}`) provisions the deep layer; or by hand: `uv venv && uv pip install -r requirements.txt` (harness SDK/MCP + IntelGraph > renderers; the collector + KB + deterministic pipeline are stdlib and need none). An explicit > `$INTEL_HOME` still overrides for a shared external KB. Full architecture, the op map, and the > evidence-envelope schema: [`connectors/intel-backend.md`](connectors/intel-backend.md). --- ## 2.5. Pivot Priority & False-Positive Control (CRITICAL) Two failure modes ruin a cluster: asserting a link that isn't there, and missing one that is. This section governs both. Apply it in Enrich, before anything reaches a report. ### Pivot priority ladder Work **down** this ladder. Never assert same-operator on a lower rung when a higher rung is available or contradicts it. Tag every asserted link in the report with the rung it rests on. | Rung | Indicator | Strength | |---|---|---| | 1 | Registrant email / phone / org — **including historic WHOIS** | decisive | | 2 | One domain carrying **two identities across its own WHOIS history** | decisive — proves an alias | | 3 | Site-verification token (Google Search Console, etc.) | decisive — proves account control | | 4 | Shared TLS certificate / SAN cross-cover | strong | | 5 | Nameserver delegation to a host the operator **runs themselves** | strong — proves zone control | | 6 | APK signing certificate | strong | | 7 | Distinctive favicon / analytics / tracker / backend tenant ID | moderate — verify below | | 8 | Co-tenancy on a **dedicated** host (few tenants) | moderate |