performing-osint-with-spiderfoot

name	performing-osint-with-spiderfoot
description	Use when automating OSINT collection using SpiderFoot REST API and CLI for target profiling, module-based reconnaissance, and structured result analysis across 200+ data sources
domain	cybersecurity
subdomain	threat-intelligence
tags	["osint","spiderfoot","reconnaissance","threat-intelligence","attack-surface","target-profiling"]
version	1.0
author	oyi77
license	Apache-2.0
nist_csf	["ID.RA-01","ID.RA-05","DE.CM-01","DE.AE-02"]

Performing OSINT with SpiderFoot

Overview

SpiderFoot is an open-source OSINT automation tool with 200+ modules that integrates with data sources for threat intelligence and attack surface mapping. This skill uses the SpiderFoot REST API and CLI (sf.py/spiderfoot-cli) to create and manage scans, select modules by use case (footprint, investigate, passive), parse structured results for domains, IPs, email addresses, leaked credentials, and DNS records, and generate target intelligence profiles.

When to Use

Trigger phrases:

• "performing osint with spiderfoot"

• "Automate OSINT collection using SpiderFoot REST API and CLI for target profiling"

• When conducting security assessments that involve performing osint with spiderfoot

• When following incident response procedures for related security events

• When performing scheduled security testing or auditing activities

• When validating security controls through hands-on testing

Prerequisites

• SpiderFoot 4.0+ installed or SpiderFoot HX cloud account
• Python 3.8+ with requests library
• SpiderFoot server running on default port 5001
• Optional: API keys for VirusTotal, Shodan, HaveIBeenPwned modules

Steps

# Example: IOC detection
import re

IOC_PATTERNS = {
    "ip": r"\b(?:\d{1,3}\.){3}\d{1,3}\b",
    "domain": r"\b[a-z0-9-]+\.[a-z]{2,}\b",
    "hash_md5": r"\b[a-f0-9]{32}\b",
    "hash_sha256": r"\b[a-f0-9]{64}\b",
}

def extract_iocs(text: str) -> dict:
    return {k: re.findall(v, text) for k, v in IOC_PATTERNS.items()}

1. Connect to SpiderFoot REST API or use CLI interface
2. Create a new scan with target specification (domain, IP, email, name)
3. Select scan modules by use case (all, footprint, investigate, passive)