ocas-scout

name	ocas-scout
description	Structured OSINT research on people, companies, and organizations. Use for provenance-backed briefs, entity resolution across public sources, background research with cited sources, or free-first research workflows that escalate to paid sources only with explicit permission. Do not use for topic research without a person/org focus (use Sift) or illegal data collection.
license	MIT
source	https://github.com/indigokarasu/scout
includes	["references/**","scripts/**"]
metadata	{"author":"Indigo Karasu (indigokarasu)","version":"4.0.0","hermes":{"category":"research","tags":["OSINT","research","people","companies","investigation","OCAS-core"]}}
tags	["OSINT","research","people","companies","investigation"]
triggers	["osint research","people research","company research","entity resolution","provenance brief"]

Interactive Menu

When invoked interactively, present a two-level menu. See references/interactive-menu.md for the full menu structure.

When to Use

• OSINT research on people, companies, or topics
• Structured investigation with source tracking
• Due diligence and background research
• Competitive intelligence gathering
• When Sift needs deeper targeted research

When NOT to Use

• General topic research (use Sift)
• Image processing (use Look)
• Knowledge graph writes (use Elephas)
• Social graph management (use Weave)

Scout

Scout conducts lawful OSINT research on people, companies, and organizations, assembling provenance-backed briefs where every claim carries a source reference, retrieval timestamp, and direct quote. It works through a tiered source waterfall — public web first, then rate-limited registries, then paid databases only with explicit permission — collecting no more than the stated research goal requires. This tiered approach exists because paid sources require explicit consent and rate-limited registries need careful quota management to avoid service degradation.

Scout integrates curated person-specific OSINT tools (theHarvester, Maigret, Holehe, h8mail, PhoneInfoga, and others), a public-records investigation framework (SEC EDGAR, USAspending, Senate lobbying, OFAC sanctions, ICIJ offshore leaks, NYC property records, OpenCorporates, CourtListener, Wayback Machine, Wikipedia/Wikidata, GDELT), and dynamically discovers new MCP-wrapped OSINT servers at runtime.

Responsibility boundary

Scout owns lawful OSINT research on people and organizations with provenance-backed output.

Scout does not own: general topic research (Sift), image processing (Look), knowledge graph writes (Elephas), social graph (Weave), communications (Dispatch).

Ontology types

Scout works with these types from spec-ocas-ontology.md:

• Entity/Person — people and their public profiles. The primary entity type Scout extracts.
• Entity/AI — AI agents or organizations when relevant to research.
• Thing/DigitalArtifact — public documents, profiles, and digital records found during research.

Scout emits Signals to Elephas after each completed research request, for each extracted entity with confidence >= med. Signal payload.type is "Person" or "AI". source_journal_type is "Research". Every emitted Signal must include a user_relevance field.

user_relevance field

Every Signal emitted by Scout carries a user_relevance field:

• "user" — the signal is relevant to the user's personal knowledge graph
• "agent_only" — agent-initiated research with no demonstrated user connection

Default is "agent_only". A signal receives user_relevance: "user" only when: (1) the user explicitly requested the research, OR (2) the entity connects to a user_relevance: "user" Chronicle entry. When in doubt, default to "agent_only".

Signal example:

{
  "signal_id": "sig-scout-20260402-001",
  "source_skill": "ocas-scout",
  "source_journal_type": "Research",
  "emitted_at": "2026-04-02T14:30:00Z",
  "user_relevance": "agent_only",
  "payload": {
    "type": "Person",
    "name": "Jane Doe",
    "confidence": "high",
    "source_refs": ["https://example.com/profile"]
  }
}

Commands

• scout.research.start — begin a new research request with subject and goal
• scout.research.expand --tier <1|2|3> — escalate to a higher source tier
• scout.brief.render — generate the final markdown brief with findings and sources
• scout.brief.render_pdf — optional PDF brief generation
• scout.status — return current research state
• scout.journal — write journal for the current run; called at end of every run
• scout.update — pull latest from GitHub source; preserves journals and data
• scout.sources.discover — discover new MCP servers relevant to current research
• scout.sources.refresh — refresh curated source lists from GitHub
• scout.sources.status — show state of dynamic source discovery

Invariants

1. Legality-first — only publicly available sources without bypassing access controls
2. Minimization — collect only what the research goal requires
3. Provenance for every claim — at least one source reference with URL, retrieval timestamp, and quote
4. Paid sources require explicit permission — Tier 3 needs a recorded PermissionGrant
5. No doxxing by default — private details suppressed unless explicitly permitted
6. Uncertainty must be surfaced — incomplete identity resolution stated clearly
7. Identity gate — a profile is verified only when 2+ seed data points overlap (name + location, etc.). Username match alone = unverified_lead; exclude from synthesis
8. Tiered verification — Sherlock results: top 3 verified immediately, 2-3 sampled, remainder only on explicit request
9. Recursion cap — recursive handle discovery allowed for 1 additional pass; hard cap at 2 Sherlock passes per request
10. Person-first tooling — always run theHarvester and OpenSanctions for person research. Run Holehe, h8mail, PhoneInfoga, Ghunt when relevant input data is available.

Input contract

ResearchRequest requires: request_id, as_of, subject (type, name, aliases, known_locations, known_handles, known_emails, known_phones), goal, constraints (time_budget_minutes, minimize_pii). Read references/scout_schemas.md for exact schema.

Research workflow

Full step-by-step workflow with tool commands, handle expansion logic, identity gating, and verification tiers is in references/scout_schemas.md.

Phase summary:

• Normalize — parse request, validate subject identity inputs
• Tier 1 collection — run person-specific tools (theHarvester, OpenSanctions) and public web search in parallel
• Handle expansion — extract handles, run Sherlock/Maigret, apply identity gate (inv 7), tiered verification (inv 8), recursion cap at 2 passes (inv 9)
• Email/phone tools — run Holehe, h8mail, EmailRep, Ghunt (if Gmail), PhoneInfoga as data is available
• Public records investigation — for company/org research or when person research reveals corporate ties, run public-records fetch scripts (SEC EDGAR, USAspending, Senate lobbying, OFAC sanctions, ICIJ offshore leaks, OpenCorporates, CourtListener, NYC ACRIS, Wayback Machine, Wikipedia/Wikidata, GDELT). See references/scout_public_records.md for source selection, execution order, and cross-reference keys.
• Entity resolution across sources — normalize names, run entity_resolution.py to cross-link entities between public-records CSVs and person-tool findings. Three match tiers: exact (high), fuzzy (medium), token_overlap (low).
• Timing correlation (optional) — run timing_analysis.py to test whether event time series (e.g., lobbying filings vs contract awards) cluster suspiciously. Permutation test, one-tailed p-value.
• Dynamic MCP discovery — if Tier 1 + public records results are thin, discover and connect to MCP-wrapped OSINT servers
• Compile — record provenance, assign confidence, escalate to Tier 2/3 only as permitted
• Brief — render markdown brief (see Output requirements). Near-match profiles surfaced explicitly, not discarded.
• Emit & journal — store findings, emit Signal files per confirmed entity (with user_relevance), write journal via scout.journal

When minimize_pii=true, suppress unnecessary sensitive details in the final brief.

Source waterfall

Read references/scout_source_waterfall.md for full tier logic.

• Tier 1 — Person-specific tools — theHarvester, OpenSanctions, Sherlock, Maigret, Holehe, h8mail, EmailRep, Ghunt, PhoneInfoga. See references/scout_person_sources.md for full list and execution order.
• Web + platform search — public web via SearchX (local SearXNG), including Twitter/X, Reddit, LinkedIn, GitHub. Always runs.
• Tier 1.5 — Public records — SEC EDGAR, USAspending, Senate lobbying (LD-1/LD-2), OFAC SDN sanctions, ICIJ offshore leaks, NYC property records (ACRIS), OpenCorporates, CourtListener, Wayback Machine, Wikipedia/Wikidata, GDELT. Python stdlib-only fetch scripts. Runs for company/org research or when person research reveals corporate ties. See references/scout_public_records.md.
• Tier 2 — rate-limited sources, registries, MCP-discovered servers. Only if enabled and useful.
• Tier 3 — paid OSINT providers, background databases. Requires explicit permission grant.

Output requirements

Markdown brief: Executive Summary, Identity Resolution Notes, Findings, Social Graph (if handles found), Digital Footprint (if email/phone tools ran), Public Records (if public-records investigation ran), Risk and Uncertainty, Source Log. Every finding carries source-backed provenance.

• Social Graph: verified profiles (platform, URL, discovery method, verification evidence) + unverified leads separately. Omit if no handles found.
• Digital Footprint: email accounts (Holehe), breaches (h8mail/HIBP), Google data (Ghunt), phone metadata (PhoneInfoga), email risk (EmailRep), dark web (OnionClaw). Omit if no data available.
• Public Records: corporate filings (SEC EDGAR), government contracts (USAspending), lobbying disclosures (Senate LD-1/LD-2), sanctions (OFAC SDN), offshore leaks (ICIJ), property records (NYC ACRIS), corporate registry (OpenCorporates), court records (CourtListener), web archives (Wayback), knowledge base (Wikipedia/Wikidata), news monitoring (GDELT). Omit if no public-records investigation was run.

Read references/scout_brief_template.md for the full template.

Inter-skill interfaces

Scout writes Signal files to Elephas (via journal signal payload). One Signal per confirmed entity or high-confidence relationship. Use schema from spec-ocas-shared-schemas.md. Every Signal must include user_relevance. See spec-ocas-interfaces.md for signal format.

Recovery Behavior

This section defines error handling and recovery procedures for all scout jobs.

Implements the recovery contract from spec-ocas-recovery.md.

• Evidence: Every run writes to {agent_root}/commons/data/ocas-scout/evidence.jsonl (including no-op runs; not_activity_reason mandatory when no side effects).
• Gap detection: On wake, checks evidence log. If gap exceeds cadence (24h update, 7d research), logs gap_detected.
• Degraded mode: When tools unavailable, logs degraded: <tool> and continues with available sources.
• Log compaction: Logs older than 30 days (no-op) or 90 days (error/gap) compacted. Last 7 days retained.

Storage layout

See references/storage-layout.md for directory structure. Read references/scout_config.md for default config.json and field descriptions.

OKRs

See references/okrs.md for skill OKR definitions.

Optional skill cooperation

• Sherlock — username-to-platform expansion during handle expansion. Falls back to sift.search("site:<platform> '<handle>'") if unavailable.
• Sift — web search during Tier 1; Sift-dork fallback when Sherlock unavailable
• Look — reverse image search for person identification when photo available
• Weave — read social graph for identity context before research (read-only)
• Elephas — emit Signal files for Chronicle promotion
• native-mcp / mcporter — connect to dynamically discovered MCP-wrapped OSINT servers
• RapidAPI — structured social media enrichment via rapidapi_call (see RapidAPI Enrichment Workflow below)

---

RapidAPI Enrichment Workflow

During Phase 3 (Handle Expansion) and Phase 4 (Email/Phone Tools), use RapidAPI social media endpoints to enrich profiles with structured data that free tools (Sherlock/Maigret) can't provide. Full procedure including person/company enrichment pipelines, rate limiting, and tier integration: see references/rapidapi-enrichment-workflow.md.

Journal outputs

• Observation Journal — research runs producing findings
• Research Journal — structured multi-source research sessions

Journals must include an entities_observed array (name, type, confidence, user_relevance). See references/journal.md for schema.

Visibility

public

Initialization

On first invocation, run scout.init: create data dirs + default config, create empty JSONL files, create journal dir, register cron jobs, log DecisionRecord, set up SearchX (SearXNG + nginx), verify person-specific tools (install missing via pip), check dark web tools (OnionClaw + Tor, non-blocking).

Background tasks

Job	Schedule	Command
scout:update	0 0 * * * (midnight daily)	scout.update
scout:research	0 9 * * 1 (Monday 9am)	scout.research
scout:sources-refresh	0 6 * * 0 (Sunday 6am)	scout.sources.refresh

Self-update

scout.update pulls the latest package from the source: URL in this file's frontmatter. Runs silently — no output unless the version changed or an error occurred.

Read references/self_update.md for the full self-update procedure.

External Catalog Review

Monitors: awesome-osint-mcp-servers (weekly), awesome-osint (monthly), API-s-for-OSINT (monthly). New tools classified by tier → added to references/scout_person_sources.md. See references/scout_mcp_discovery.md for dynamic discovery. See references/sources-refresh.md for the concrete refresh procedure that runs on the scout:sources-refresh cron.

Gotchas

• Tier 3 sources require explicit permission — Paid OSINT providers and background databases (Tier 3) cannot be queried without a recorded PermissionGrant. The skill will silently skip Tier 3 even if credentials are configured.
• User relevance defaults to agent_only — Signals emitted to Elephas default to user_relevance: "agent_only" unless the user explicitly requested the research or the entity connects to a known user Chronicle entry. This means most Scout entities won't be promoted.
• Hard cap at 2 Sherlock passes — Recursive handle discovery is allowed for exactly 1 additional pass (2 total). Further recursion is silently blocked regardless of leads found.
• Identity gate requires 2+ data points — A username match alone produces only unverified_lead status. Profiles are verified only when 2+ seed data points overlap (name + location, etc.).
• minimize_pii suppresses home addresses and personal details — When the user sets minimize_pii=true, the final brief suppresses unnecessary sensitive details even if they were found during research. Re-run without the flag to see full data.
• "Not subscribed" ≠ broken on RapidAPI. Many APIs return tools on tools/list but "You are not subscribed to this API" on actual calls. These are tier-gated by the provider, not broken. 3 genuinely broken LinkedIn APIs (linkedin-data-api, linkedin-api8, li-data-scraper) — provider shut down. 4 new ones work: fresh-linkedin-scraper-api (primary), linkedinscraper, linkedin-email-finder5, linkedin-lead-enrichment.
• gh api base64 content is unreliable for counting — gh api repos/<owner>/<repo>/readme --jq '.content' | base64 -d can silently return truncated or empty results. For fetching raw GitHub README content, use curl -s "https://raw.githubusercontent.com/<owner>/<repo>/<branch>/README.md" instead. This returns clean text that wc -l and grep can process directly.
• Branch names vary across repos — jivoi/awesome-osint uses master (not main). Always verify branch name with a quick curl -s -o /dev/null -w "%{http_code}" probe before fetching content.
• curl timeout for unreliable endpoints — Some GitHub endpoints return 0 bytes under rate-limiting. Use --max-time 15 and check HTTP status code before processing output.
• Public-records scripts use stdlib only — All fetch scripts in scripts/fetch_*.py use Python stdlib only (no pip installs required). They use urllib.request with retry logic and respect Retry-After headers. 429 responses surface immediately with the upstream's quota message.
• Entity resolution is token-bag only — entity_resolution.py does NOT use external fuzzy libraries (no rapidfuzz, no jellyfish). Token-bag matching is the upper bound. For Levenshtein, transliteration, or phonetic matching, pip-install separately.
• ICIJ offshore leaks cache — First run downloads ~70 MB bulk CSV. Cached for 30 days under $HERMES_OSINT_CACHE/icij/ (default: ~/.cache/hermes-osint/icij/). Subsequent runs search the local cache.
• Statistical significance ≠ wrongdoing — timing_analysis.py p < 0.05 means the timing pattern is unlikely under the null. It does not establish corruption. Always state this in the brief.

Support File Map

File	When to read
references/scout_schemas.md	Before requests/findings/briefs; full workflow steps
references/scout_config.md	Default config.json and field descriptions
references/scout_source_waterfall.md	Before tier selection or escalation
references/scout_brief_template.md	Before rendering briefs
references/scout_person_sources.md	At start of every person research run
references/scout_public_records.md	At start of every company/org research run; when person research reveals corporate ties; before Phase 5
references/scout_mcp_discovery.md	Before scout.sources.discover; before Tier 2 escalation
references/sources-refresh.md	Before scout.sources.refresh; before weekly cron refresh runs
references/rapidapi-osint-params.md	Before RapidAPI enrichment; param patterns per platform
references/rapidapi-enrichment-workflow.md	During Phase 3/4 research; RapidAPI person/company enrichment pipelines
references/journal.md	Before scout.journal; at end of every run
references/self_update.md	Before running scout.update; when debugging self-update failures

Update command

This skill self-updates every 24 hours via:

scout.update

This pulls the latest version from GitHub and restarts the skill's background tasks if applicable.