openosint-ai-osint-framework

# OpenOSINT AI OSINT Framework > Skill by [ara.so](https://ara.so) — Security Skills collection. OpenOSINT is an AI-powered Open Source Intelligence framework that combines 11 OSINT tools into a unified interface. It operates as an interactive REPL with natural language investigation, a direct CLI for scripting, and an MCP server for AI client integration. The AI agent intelligently chains tools based on findings and compiles structured reports. All tools run as async subprocess wrappers with hard timeout enforcement. ## Installation ```bash # Clone and install git clone https://github.com/OpenOSINT/OpenOSINT.git cd OpenOSINT pip install -e . # Set Anthropic API key (or use --provider ollama for local models) export ANTHROPIC_API_KEY=sk-ant-your-key-here # Install external OSINT tools pip install holehe sherlock-project sublist3r # Download phoneinfoga binary from https://github.com/sundowndev/phoneinfoga/releases # Place in PATH ``` ## Optional Dependencies ```bash # For local LLM (no API key required) pip install ollama ollama pull llama3.2 # For Shodan API pip install shodan export SHODAN_API_KEY=your-shodan-key # For PDF reports pip install reportlab # API keys for enhanced functionality export HIBP_API_KEY=your-hibp-key # HaveIBeenPwned export IPINFO_TOKEN=your-ipinfo-token # ipinfo.io export VIRUSTOTAL_API_KEY=your-vt-key # VirusTotal ``` ## Usage Modes ### Interactive REPL (AI-Powered) ```bash # Launch interactive AI agent openosint # Or explicitly openosint shell # Use local Ollama instead of Anthropic openosint --provider ollama ``` In REPL, type natural language queries: ``` openosint ❯ investigate user@example.com openosint ❯ find accounts for johndoe99 openosint ❯ check breaches for admin@company.com openosint ❯ enumerate subdomains of example.com ``` REPL commands: - `<target>` - Investigate email/username/domain/IP/phone - `clear` - Reset conversation memory - `save` - Save last report to reports/ - `tools` - List available tools and status - `config` - Show current configuration - `help` - Show all commands - `exit` / Ctrl-D - Exit ### Direct CLI (No AI) ```bash # Email investigation openosint email target@example.com openosint email target@example.com -t 60 # 60 second timeout # Username search openosint username johndoe99 openosint username johndoe99 -t 120 # Shodan lookup openosint shodan 8.8.8.8 openosint shodan "apache port:80 country:DE" # VirusTotal lookup openosint virustotal example.com openosint virustotal 192.168.1.1 # Multi-target investigation openosint multi "user@example.com johndoe99 example.com" # Parallel execution (run multiple tools simultaneously) openosint --parallel email user@example.com openosint --parallel username johndoe99 # JSON output for scripting openosint --json email user@example.com ``` ## Python API Usage ### Direct Tool Usage ```python import asyncio from openosint.tools.email import search_email from openosint.tools.username import search_username from openosint.tools.breach import search_breach from openosint.tools.whois import search_whois from openosint.tools.ip import search_ip from openosint.tools.domain import search_domain from openosint.tools.dorks import generate_dorks from openosint.tools.paste import search_paste from openosint.tools.phone import search_phone from openosint.tools.shodan import search_shodan from openosint.tools.virustotal import search_virustotal async def investigate_email(email: str): """Investigate an email address.""" # Search social accounts accounts = await search_email(email, timeout=60) print(f"Social accounts: {accounts}") # Check data breaches breaches = await search_breach(email) print(f"Breaches: {breaches}") # Search paste dumps pastes = await search_paste(email) print(f"Paste dumps: {pastes}") # Generate search dorks dorks = await generate_dorks(email) print(f"Google dorks: {dorks}") async def investigate_username(username: str): """Find username across platforms.""" platforms = await search_username(username, timeout=120) print(f"Found on platforms: {platforms}") async def investigate_domain(domain: str): """Investigate a domain.""" # WHOIS lookup whois = await search_whois(domain) print(f"WHOIS: {whois}") # Subdomain enumeration subdomains = await search_domain(domain) print(f"Subdomains: {subdomains}") # VirusTotal check vt_result = await search_virustotal(domain) print(f"VirusTotal: {vt_result}") async def investigate_ip(ip: str): """Investigate an IP address.""" # IP geolocation ip_info = await search_ip(ip) print(f"IP info: {ip_info}") # Shodan lookup shodan_data = await search_shodan(ip) print(f"Shodan: {shodan_data}") async def investigate_phone(phone: str): """Investigate a phone number (E.164 format).""" phone_info = await search_phone(phone) print(f"Phone info: {phone_info}") # Run investigations asyncio.run(investigate_email("target@example.com")) asyncio.run(investigate_username("johndoe99")) asyncio.run(investigate_domain("example.com")) asyncio.run(investigate_ip("8.8.8.8")) asyncio.run(investigate_phone("+14155552671")) ``` ### AI Agent Usage ```python import asyncio from openosint.agent import OpenOSINTAgent async def ai_investigation(): """Run AI-powered investigation.""" agent = OpenOSINTAgent( api_key=None, # Uses ANTHROPIC_API_KEY from env provider="anthropic", # or "ollama" model="claude-3-5-sonnet-20241022" ) # Natural language investigation response = await agent.investigate("investigate user@example.com") print(response) # Agent automatically chains tools based on findings response = await agent.investigate( "find all accounts for johndoe99 and check for breaches" ) print(response) # Get conversation history history = agent.get_history() # Clear conversation agent.clear_history() asyncio.run(ai_investigation()) ``` ### Parallel Execution ```python import asyncio from openosint.tools.email import search_email from openosint.tools.breach import search_breach from openosint.tools.paste import search_paste async def parallel_email_investigation(email: str): """Run multiple tools in parallel.""" results = await asyncio.gather( search_email(email, timeout=60), search_breach(email), search_paste(email), return_exceptions=True # Don't fail if one tool errors ) accounts, breaches, pastes = results report = { "email": email, "accounts": accounts if not isinstance(accounts, Exception) else str(accounts), "breaches": breaches if not isinstance(breaches, Exception) else str(breaches), "pastes": pastes if not isinstance(pastes, Exception) else str(pastes) } return report # Run parallel investigation result = asyncio.run(parallel_email_investigation("target@example.com")) print(result) ``` ## Tool-Specific Examples ### Email Investigation ```python async def comprehensive_email_scan(email: str): """Complete email OSINT scan.""" from openosint.tools.email import search_email from openosint.tools.breach import search_breach from openosint.tools.paste import search_paste from openosint.tools.dorks import generate_dorks print(f"[*] Investigating {email}") # Find social accounts print("[*] Searching social accounts...") accounts = await search_email(email) # Check breaches (requires HIBP_API_KEY) print("[*] Checking data breaches...") breaches = await search_breach(email) # Search paste dumps print("[*] Searching paste sites...") pastes = await search_paste(email) # Generate Google dorks print("[*] Generating Google dorks...") dorks = await generate_dorks(email) return { "accounts": accounts, "breaches": breaches, "pastes": pastes, "dorks": dorks } ``` ### Username Investigation ```python async def comprehensive_username_scan(username: str): """Complete username OSINT scan.""" from openosint.tools.username import search_username from openosint.tools.paste import search_paste from openosint.tools.dorks import generate_dorks print(f"[*] Investigating {username}") # Search across 300+ platforms print("[*] Searching platforms (this may take 2+ minutes)...") platforms = await search_username(username, timeout=180) # Search paste dumps print("[*] Searching paste sites...") pastes = await search_paste(username) # Generate Google dorks print("[*] Generating Google dorks...") dorks = await generate_dorks(username) return { "platforms": platforms, "pastes": pastes, "dorks": dorks } ``` ### Domain Investigation ```python async def comprehensive_domain_scan(domain: str): """Complete domain OSINT scan.""" from openosint.tools.whois import search_whois from openosint.tools.domain import search_domain from openosint.tools.virustotal import search_virustotal from openosint.tools.dorks import generate_dorks print(f"[*] Investigating {domain}") # WHOIS lookup print("[*] Running WHOIS...") whois = await search_whois(domain) # Subdomain enumeration print("[*] Enumerating subdomains...") subdomains = await search_domain(domain) # VirusTotal scan (requires VIRUSTOTAL_API_KEY) print("[*] Checking VirusTotal...") vt_result = await search_virustotal(domain) # Generate Google dorks print("[*] Generating Google dorks...") dorks = await generate_dorks(domain) return { "whois": whois, "subdomains": subdomains, "virustotal": vt_result, "dorks": dorks } ``` ### IP Investigation ```python async def comprehensive_ip_scan(ip: str): """Complete IP OSINT scan.""" from openosint.tools.ip import search_ip from openosint.tools.shodan import search_shodan from openosint.tools.virustotal import search_virustotal print(f"[*] Investigating {ip}") # IP geolocation print("[*] Geolocating IP...") ip_info = await search_ip(ip) # Shodan lookup (requires SHODAN_API_KEY) print("[*] Querying Shodan...") shodan_data = await search_shodan(ip) # VirusTotal scan (requires VIRUSTOTAL_API_KEY) print("[*] Checking VirusTotal...") vt_result = await search_virustotal(ip) return { "ip_info": ip_info, "shodan": shodan_data, "virustotal": vt_result } ``` ### Shodan Advanced Queries ```python async def shodan_queries(): """Advanced Shodan search examples.""" from openosint.tools.shodan import search_shodan # Host lookup (specific IP) host = await search_shodan("8.8.8.8") # Banner search apache = await search_shodan("apache port:80") # Country-specific search german_servers = await search_shodan("apache port:80 country:DE") # Vulnerable systems heartbleed = await search_shodan("vuln:CVE-2014-0160") # ICS/SCADA scada = await search_shodan("port:502") # Modbus # Webcams cameras = await search_shodan("Server: SQ-WEBCAM") return { "host": host, "apache": apache, "german_servers": german_servers, "heartbleed": heartbleed, "scada": scada, "cameras": cameras } ``` ## MCP Server Integration OpenOSINT can run as an MCP server for Claude Desktop or other MCP clients: ```json { "mcpServers": { "openosint": { "command": "python", "args": ["-m", "openosint.mcp_server"], "env": { "ANTHROPIC_API_KEY": "your-key", "HIBP_API_KEY": "your-key", "SHODAN_API_KEY": "your-key", "VIRUSTOTAL_API_KEY": "your-key", "IPINFO_TOKEN": "your-token" } } } } ``` Then in Claude Desktop, you can use natural language: ``` "Investigate user@example.com for social accounts and data breaches" "Find all platforms where johndoe99 has accounts" "Enumerate subdomains for example.com" ``` ## Configuration ### Environment Variables ```bash # Required for AI agent (unless using --provider ollama) export ANTHROPIC_API_KEY=sk-ant-your-key # Optional API keys export HIBP_API_KEY=your-hibp-key # For search_breach export SHODAN_API_KEY=your-shodan-key # For search_shodan export VIRUSTOTAL_API_KEY=your-vt-key # For search_virustotal export IPINFO_TOKEN=your-ipinfo-token # For enhanced search_ip # For local LLM export OLLAMA_HOST=http://localhost:11434 # Default Ollama endpoint ``` ### Timeouts All tools accept a `timeout` parameter (seconds): ```python # Default timeouts await search_email(email, timeout=60) # 60 seconds await search_username(username, timeout=120) # 120 seconds (sherlock is slow) await search_domain(domain, timeout=90) # 90 seconds await search_phone(phone, timeout=30) # 30 seconds await search_shodan(query, timeout=30) # 30 seconds # Custom timeout await search_email(email, timeout=180) # 3 minutes ``` ## Common Patterns ### Batch Email Investigation ```python