osint-due-diligence

name	osint-due-diligence
description	Use when conducting OSINT due diligence on companies or individuals, performing background investigations, verifying identities, assessing business partners, or investigating any entity using open-source intelligence. Also use for third-party vendor cyber risk and data protection assessments (GDPR/NIS2/DORA compliance). Orchestrates 30+ CLI OSINT tools through a structured 8-phase company, 7-phase individual, or 6-phase vendor risk investigation methodology based on Berkeley Protocol, Bellingcat, and IntelTechniques standards.
version	1.2.0
author	Hermes Agent (Mbaunguraije Tjikuzu)
license	MIT
metadata	{"hermes":{"tags":["osint","due-diligence","investigation","reconnaissance","background-check","intelligence","verification"],"related_skills":["browser-research-workflows","academic-research-workflows","himalaya"]}}

OSINT Due Diligence

Orchestrate open-source intelligence investigations on companies and individuals using CLI tools with a structured methodology grounded in professional standards (Berkeley Protocol, Bellingcat, IntelTechniques).

Overview

This skill turns Hermes Agent into a due diligence investigator. It provides:

1. A phased methodology for company (8 phases) and individual (7 phases) investigations — what to look for, in what order, from which sources.
2. A curated toolkit of 30+ CLI-first OSINT tools organized by category — which tool to reach for at each phase.
3. A reporting framework that produces structured, evidence-backed due diligence reports with source documentation, red flags, and risk ratings.
4. Red flag catalogs for both company and individual investigations so nothing is missed.

The methodology traces to the Berkeley Protocol (UN-backed international standard), Bellingcat's investigative journalism standards, and Michael Bazzell's IntelTechniques practitioner framework. See references/methodology.md for the full academic grounding.

When to Use

• Investigating a company before a business partnership, investment, or vendor relationship
• Third-party vendor risk assessment — cyber security, data protection, GDPR/NIS2/DORA compliance
• Assessing a vendor/processor before signing a DPA or data-sharing agreement
• Background-checking an individual (hire, board appointment, counter-party)
• Verifying claims made by an entity (credentials, history, associations)
• Assessing sanctions, PEP, or regulatory exposure
• Mapping an entity's digital footprint and breach exposure
• Completing a TPRM (Third-Party Risk Management) questionnaire or security review
• Any scenario where the user says "look into," "check out," "due diligence," "vendor assessment," "supplier risk," "background check," "investigate," "vet this vendor," or "what do we know about"

Don't use for: Criminal investigations requiring legal authority, accessing paywalled/classified data without authorization, or replacing legal counsel's formal DD process. This skill uses publicly available information only.

Prerequisites

Install the core toolset before the first investigation:

# Python tools (pip-installable)
pip install sherlock-project maigret holehe theHarvester spiderfoot shodan h8mail recon-ng instaloader toutatis

# Go-based tools
go install github.com/owasp-amass/amass/v4/...@master
go install github.com/projectdiscovery/subfinder/v2/cmd/subfinder@latest

# Optional: Docker-based tools for heavy automation
# docker pull smicallef/spiderfoot   (if not using pip version)
# docker pull yogeshojha/rengine      (full recon framework with web UI)

See scripts/install-tools.sh for a one-shot installation script.

API keys to configure (optional but recommended):

• Have I Been Pwned (HIBP) API key — breach checking
• Shodan API key — device/infrastructure search
• DeHashed API key — breach data search
• OpenCorporates API key — company registry queries (higher rate limits)

Investigation Workflows

Company Due Diligence (8 Phases)

Execute phases in order. Each phase builds on the previous. Skip phases only when explicitly told to.

Phase 1: Corporate Identity & Structure Goal: Confirm the entity legally exists and understand its structure.

theHarvester -d <domain> -b all                    # emails, subdomains, names
curl -s "https://api.opencorporates.com/v0.4/companies/search?q=<company>" | python3 -m json.tool
# Check national business registry (Companies House, SEC EDGAR, etc.)
spiderfoot-cli -s <domain> -t DOMAIN_NAME,EMAIL_ADDRESS,HUMAN_NAME

What to record: legal name, registration number, jurisdiction, date of incorporation, legal structure, officers/directors, filing history.

Phase 2: Financial Health Goal: Assess financial stability, funding, and red flags.

# Search SEC EDGAR for public filings
# Check Crunchbase/PitchBook for funding rounds
# Look for tax liens, judgments, bankruptcies via court records

Sources: SEC EDGAR, annual reports, D&B, Experian, court records, Crunchbase. Use the browser for registries without APIs.

Phase 3: Regulatory & Compliance Goal: Screen for sanctions, watchlists, and regulatory actions.

# Check OFAC SDN list
curl -s "https://sanctionslistservice.ofac.treas.gov/api/Search" # requires setup
# Check industry-specific regulators (FINRA, FDA, EPA, FTC)

What to record: sanctions matches, PEP status, regulatory enforcement history, license validity.

Phase 4: Litigation & Legal History Goal: Uncover lawsuits, criminal proceedings, regulatory enforcement.

# Browser-based: PACER (US), state court databases, LexisNexis, Westlaw
# For non-US: national court databases, EUR-Lex, BAILII

Phase 5: Reputation & Media Goal: Assess public perception, controversies, customer sentiment.

# Browser: Google News, Factiva, Glassdoor, BBB, TrustPilot
# Social media monitoring

Phase 6: Operations & Supply Chain Goal: Understand physical presence, scale, products, IP.

shodan domain <domain>                               # exposed infrastructure
amass enum -d <domain>                               # subdomain mapping
subfinder -d <domain>                                # passive subdomain enum

What to record: locations, employee count via LinkedIn, IP portfolio (USPTO), major customers/suppliers.

Phase 7: Digital Footprint & Cybersecurity Goal: Map online presence, breach history, security posture.

curl -s "https://haveibeenpwned.com/api/v3/breacheddomain/<domain>" -H "hibp-api-key: $HIBP_KEY"
h8mail -t <email>                                    # breach hunting
shodan host <IP>                                     # exposed services
# Check Wayback Machine for historical website changes

What to record: domains, breach exposure, exposed services, historical web presence.

Phase 8: Beneficial Ownership & Network Analysis Goal: Identify who really owns/controls the entity.

curl -s "https://api.opencorporates.com/v0.4/companies/<jurisdiction>/<id>/statements"
# Search ICIJ Offshore Leaks database
# Search OCCRP database

What to record: UBOs, offshore structures, political connections, related entities.

Individual Due Diligence (7 Phases)

Phase 1: Identity Verification — Confirm name, aliases, DOB, nationality via public records.

Phase 2: Professional History — LinkedIn, SEC filings (board roles), Google Scholar, USPTO patents.

theHarvester -d <domain> -b linkedin
# Browser: LinkedIn profile analysis

Phase 3: Financial Profile — Property records, bankruptcy, tax liens, sanctions.

Phase 4: Legal & Criminal History — Court databases, sex offender registries, professional discipline.

Phase 5: Digital Footprint — Social media, usernames, emails, breach exposure.

sherlock <username>                                  # 400+ social networks
maigret <username> --all-sites                       # 3000+ sites, detailed dossier
holehe <email>                                       # 120+ site registration check
h8mail -t <email>                                    # breach/leak search
ghunt email <email>                                  # Google account investigation

Phase 6: Associations & Network — Business partners, family, political donations, social graph.

Phase 7: Reputation & Media — News mentions, social sentiment, controversy, awards.

Fuzzy Search & Partial Information

Real investigations rarely start with complete, verified data. Use these strategies when you have ambiguous, partial, or uncertain input.

Step 0: Assess what you actually have. Before running any tools, categorize the input:

• Exact & verified — full legal name, confirmed domain, known email → proceed with standard phases
• Partial — first name only, company without jurisdiction, username without platform → generate variants first
• Ambiguous — common name (John Smith), name in non-Latin script, nickname, typo possible → widen search, then narrow
• Single data point — only a phone number, only an email, only a domain → pivot from that point

Name variant generation. For individual names, generate all plausible variants before searching:

# Use the variant generator script
python3 scripts/generate-variants.py "Robert J. Smith"

# Manual strategy if script unavailable:
# 1. Full formal:     Robert James Smith
# 2. First + Last:    Robert Smith
# 3. Initial + Last:  R. Smith, R Smith, RJ Smith
# 4. Nicknames:       Bob Smith, Bobby Smith, Rob Smith
# 5. Reversed:        Smith Robert, Smith R
# 6. Common typos:    Robert Smyth, Robert Smithe
# 7. Hyphenated:      Robert James-Smith
# 8. Middle name as first: James Smith

Run Sherlock/Maigret with the top 3-5 most likely variants in parallel. Cross-reference results — matching profile photos or consistent bios across variants increase confidence.

Company name variant strategy. Legal names differ from trading names:

# Example: "Acme Corp" could be registered as:
# - Acme Corporation
# - Acme Corp LLC
# - Acme Corp Ltd
# - Acme Corporate
# - The Acme Corporation
# - Acme (without suffix)

# Search strategy:
# 1. Exact match on OpenCorporates
# 2. Fuzzy/starts-with search on OpenCorporates
# 3. Domain WHOIS lookup (acme.com → find registrant name)
# 4. LinkedIn search for company page
# 5. Trademark database search (USPTO/WIPO)

Phonetic matching. When you only know how a name sounds (heard verbally, transliterated from non-Latin script):

# Soundex/Metaphone variants for "Mohammed":
# Muhammad, Mohammad, Mohamed, Muhammed, Mohamad, Muhamed
# Strategy: generate all common transliterations, search all of them

# Use the variant script with --phonetic flag:
python3 scripts/generate-variants.py "Mohammed" --phonetic

Pivoting from a single data point:

You have	Pivot to find
Email address	Username (part before @), domain → company, holehe registered sites
Phone number	Carrier lookup → country/city → name databases, WhatsApp/Telegram check
Domain name	WHOIS → registrant, theHarvester → emails, Amass → subdomains
Username	Maigret/Sherlock → other profiles → real name, email, location
Profile photo	Reverse image search → other accounts, original source
Physical address	Property records → owner, Google Maps → business at location

When to widen vs narrow:

• Widen when: zero results on initial search, common name (need to find the right one), entity may have rebranded
• Narrow when: too many results (add location, age range, employer), found probable match (verify with additional data points)
• Stop widening when: you have 5-15 candidate matches to verify
• Stop narrowing when: you have 1-3 high-confidence matches with 2+ corroborating data points each

Uncertainty tracking. When input is fuzzy, tag every finding with confidence:

• [Confirmed] — 2+ independent sources agree, no contradictions
• [Probable] — 1 strong source + circumstantial support, or 2 sources with minor inconsistency
• [Possible] — 1 source only, or plausible but unverified
• [Unverified] — Lead only, needs follow-up investigation
• [Disconfirmed] — Was a candidate but ruled out by contradictory evidence

Multi-wave search strategy for common names:

Wave 1: Name + known location (city/country)
Wave 2: Name + known employer/school
Wave 3: Name + known username/email
Wave 4: Cross-reference waves 1-3 — which person appears in multiple?
Wave 5: Deep-dive on the intersecting candidate(s)

Never present a common-name search result as definitive without corroborating data points (location, employer, photo match, consistent bio details).

Third-Party Vendor Cyber Risk & Data Protection Assessment

Use this workflow when vetting a vendor, processor, or technology partner before sharing data, integrating systems, or signing a contract. This is the compliance-grade variant of company DD — designed for DPA (Data Protection Agreement) due diligence, GDPR Article 28 assessments, and cyber supply chain risk management.

When to use this variant instead of standard company DD:

• Vendor will process personal data on your behalf (GDPR processor assessment)
• Vendor will connect to your network/systems (cyber supply chain risk)
• Vendor will host or store your data (cloud/SaaS assessment)
• Regulatory requirement (GDPR, NIS2, DORA, PCI DSS, HIPAA, POPIA)
• You're completing a vendor security questionnaire or TPRM (Third-Party Risk Management) assessment

Phase V1: Data Protection & Privacy Posture

Goal: Understand what data the vendor handles, where, and under what legal framework.

Checklist:
- What personal data categories does the vendor process? (PII, health, financial, minors, etc.)
- Where is data stored/processed? (data residency — specific countries/regions)
- Does the vendor transfer data internationally? If so, under what mechanism? (SCCs, BCRs, adequacy decision)
- Does the vendor have a DPA (Data Protection Agreement) or DPA template available?
- Does the vendor use subprocessors? If so, who are they and where are they?
- What is the vendor's breach notification timeline? (GDPR requires 72 hours)
- Privacy policy: clear, accessible, GDPR-compliant? Or vague and buried?
- Cookie/tracking consent mechanism on their website (basic hygiene check)

Key sources:
- Vendor's privacy policy and terms of service
- Vendor's DPA (if published)
- Subprocessor list (required by GDPR Art. 28 for processors)
- ICO/DPA register (UK), CNIL (France), or local DPA public registers

Phase V2: Security Certifications & Assurance

Goal: Verify the vendor's security claims with independent evidence.

Checklist:
- SOC 2 Type II report — request and review (not just "SOC 2 compliant" claim)
- ISO 27001 certificate — verify on the certification body's public register
- PCI DSS compliance — if handling payment card data, verify on VISA/Mastercard registers
- HIPAA compliance — if handling US health data, request BAA (Business Associate Agreement)
- FedRAMP — if US government, verify on the FedRAMP marketplace
- Cyber Essentials / Cyber Essentials Plus (UK)
- Penetration test summary — when was the last test? By whom? Remediation status?
- Bug bounty or vulnerability disclosure program — does it exist? Is it responsive?

Key sources:
- Certification body public registers (ISO 27001: look up the cert number on the CB's site)
- Vendor's trust/security page
- SOC 2 report (request directly — not publicly available)
- Bug bounty platforms: HackerOne, Bugcrowd

Phase V3: Breach & Incident History

Goal: Has this vendor been breached? How did they handle it?

h8mail -t @<vendor_domain>                            # employee emails in breaches
curl -s "https://haveibeenpwned.com/api/v3/breaches" | grep -i "<vendor>"
# Search: "<vendor> data breach" "<vendor> security incident"
# Search: "<vendor> ransomware" "<vendor> leak"

Checklist:
- Any publicly disclosed breaches? When? What data? Root cause?
- Any regulatory fines or enforcement actions? (ICO, CNIL, FTC, state AGs)
- Any class action lawsuits related to data breaches or privacy violations?
- Any dark web mentions of the vendor's data? (ransomware leak sites, forums)
- How did the vendor respond to past incidents? Transparent or evasive?
- Any past-due security patches or end-of-life software mentioned in breach reports?

Key sources:
- Have I Been Pwned (domain search)
- State AG data breach notifications (US)
- ICO enforcement action register (UK)
- FTC privacy enforcement cases
- Google News search for breach history
- Ransomware leak site monitoring (Ransomlook, DarkFeed)

Phase V4: Technical Security Posture

Goal: Assess the vendor's externally visible security hygiene.

shodan domain <vendor_domain>                          # exposed services
amass enum -d <vendor_domain>                          # subdomain discovery
subfinder -d <vendor_domain>                            # passive subdomain enum

# Check TLS/SSL configuration
curl -sI https://<vendor_domain> | grep -i strict-transport-security

# Check DMARC/SPF/DKIM (email spoofing protection)
dig TXT _dmarc.<vendor_domain>
dig TXT <vendor_domain> | grep spf

Checklist:
- HTTPS enforced everywhere? HSTS header present?
- TLS version: still supporting TLS 1.0/1.1? (should be 1.2+)
- Exposed services: RDP, MySQL, Redis, MongoDB open to the internet?
- Email security: DMARC, SPF, DKIM configured? DMARC policy at least "p=quarantine"?
- Subdomain hygiene: forgotten/test/dev subdomains exposing internal tools?
- Cloud storage: any public S3 buckets or Azure blobs? (search: site:bucketname.s3.amazonaws.com)
- Security.txt file present? (https://<domain>/.well-known/security.txt)
- CSP (Content Security Policy) header present?

Key sources:
- Shodan, Censys for exposed services
- SSL Labs (ssllabs.com) for TLS configuration
- MXToolbox for DMARC/SPF/DKIM checks
- GrayHatWarfare for public cloud storage
- urlscan.io for historical page scans

Phase V5: Business Continuity & Operational Resilience

Goal: Will this vendor survive an incident? Do they have recovery plans?

Checklist:
- Business continuity / disaster recovery plan — do they have one? Tested?
- SLA uptime commitments — what's guaranteed? What's the penalty?
- Data backup frequency and retention — RPO (Recovery Point Objective) and RTO (Recovery Time Objective)?
- Data portability and exit strategy — how do you get your data OUT if you leave?
- Vendor lock-in risk — proprietary formats? API access? Export tools?
- Financial stability — could this vendor go bankrupt? (Phase 2 of company DD)
- Key person risk — small vendor dependent on 1-2 people?
- Geographic concentration risk — all infrastructure in one region?

Key sources:
- Vendor's SLA and terms of service
- Vendor's security/trust page
- Company DD Phase 2 (Financial Health)
- Vendor's status page (incident history, uptime transparency)

Phase V6: Regulatory & Contractual Alignment

Goal: Ensure the vendor contract covers your compliance obligations.

Checklist:
- Does the DPA cover: data categories, processing purposes, data subjects, retention?
- Does the contract include: audit rights, breach notification timelines, subprocessor change notification?
- Does the vendor agree to your applicable data protection law? (GDPR, POPIA, LGPD, CCPA, etc.)
- Liability caps — are they proportionate to the data risk?
- Insurance — does the vendor carry cyber insurance? What coverage?
- Termination assistance — does the contract require the vendor to help migrate data?
- Jurisdiction and governing law — favorable to you or the vendor?

Key sources:
- Vendor's standard terms / DPA
- Your organization's procurement/legal template
- Regulatory guidance (ICO, EDPB, local DPA)

Vendor Cyber Risk Red Flags — Quick Reference

Critical (stop and escalate):

• No DPA available or refuses to sign one
• Data stored/processed in jurisdictions without adequate data protection
• History of unremediated breaches or regulatory fines for data protection
• No breach notification commitment (or timeline exceeds 72 hours)
• Refuses to disclose subprocessors
• Exposed database/admin panels on public internet
• No DMARC/SPF configured (email spoofable)
• Company appears to be a shell/brass plate (Phase 1 DD failure)

High (require remediation before contract):

• SOC 2 / ISO 27001 claimed but cannot produce certificate/report
• TLS 1.0/1.1 still supported
• No HSTS header
• No security.txt or vulnerability disclosure program
• DMARC policy at "p=none" (monitoring only, not protecting)
• Past breaches with unclear remediation
• Weak password policy or no MFA support
• Single-region infrastructure with no DR plan

Medium (document and monitor):

• Privacy policy vague on data retention/deletion
• No published subprocessor list (but provides on request)
• No public bug bounty program
• SPF record too permissive (+all or ~all without strict enforcement)
• Vendor is small/new with limited track record
• No cyber insurance (or unknown)

See references/vendor-risk-assessment.md for the full vendor risk assessment template with scoring matrix.

For rapid lookups without the full methodology:

# Domain: what does a company expose?
theHarvester -d <domain> -b google,linkedin,shodan

# Person: where does this username exist?
maigret <username> --no-progressbar --json output.json

# Email: is it in breaches? what sites is it registered on?
h8mail -t <email>; holehe <email>

# Infrastructure: what's exposed?
shodan domain <domain>; amass enum -d <domain>

# Phone: carrier, location, online footprints?
phoneinfoga scan -n <phone_number>

Tool Reference

Complete catalog with 30 tools across 6 categories: references/tool-catalog.md

Category	Primary Tool	Secondary	API Available
People search	Maigret (3000+ sites)	Sherlock (400+ networks)	No
Company DD	SpiderFoot (200+ modules)	theHarvester	SpiderFoot REST API
Domain recon	Amass, Subfinder	Shodan CLI, Web-Check	Shodan API
Social media	Instaloader (IG API)	Holehe (email→sites)	Instaloader Python lib
Breach data	H8mail	Mosint	HIBP, DeHashed APIs
Automation	Recon-ng	Osmedeus (LLM-powered)	Recon-ng modules API

Report Generation

After completing the investigation phases, produce a structured report. Use references/report-template.md for the full 11-section template.

Minimum report structure:

1. Executive Summary (subject ID, scope, overall risk rating, key findings, recommended actions)
2. Subject Identification
3. Scope and Methodology
4. Findings by phase (Corporate/Individual Profile, Financial, Regulatory, Legal, Reputation, Digital, Network)
5. Red Flags and Risk Assessment (categorized by severity with sources)
6. Conclusions and Recommendations
7. Appendices (source documentation, search query log, chain of custody)

Risk ratings: Low / Medium / High / Critical

Write the report to a markdown file at the path the user specifies, or default to ~/osint-reports/<subject>-<date>.md.

Common Pitfalls

1. Skipping the methodology. Running tools randomly without the phase structure produces scattered, non-actionable results. Follow the phases in order — each builds context for the next.

2. Single-source conclusions. Professional standards require minimum 2 independent sources. A username match on one site is a lead, not a finding.

3. Not documenting sources. Every claim in the report must trace back to a specific source with URL and access date. Without this, the report has no credibility.

4. Over-relying on automated tools. Tools miss things. After each automated pass, spend 5 minutes on manual browser verification of the top findings.

5. Ignoring jurisdiction. Company registries, court systems, and public records vary by country. A "clean" report that only checked US databases for a BVI entity is incomplete.

6. Not dating information. A breach from 2019 is different from a breach last week. Always timestamp findings.

7. Confusing absence of evidence with evidence of absence. If a tool finds nothing, say "no results found via [tool]" — not "clean" or "no issues."

8. API rate limits. HIBP, Shodan, OpenCorporates all have rate limits. Batch queries with sleep intervals. Maigret and Sherlock are rate-limited by target sites, not by the tool itself — they can be slow (10-30 min for full scan).

9. h8mail requires API keys. It aggregates HIBP, DeHashed, SnusBase etc. — configure API keys in ~/.h8mail.ini before use.

10. Sherlock vs. Maigret. Sherlock checks 400+ sites for exact username matches (fast, broad). Maigret checks 3000+ sites and builds a detailed dossier with profile photos/bios (slower, deeper). Use Sherlock for quick presence check, Maigret for deep investigation.

Verification Checklist

After every investigation, verify:

• All phases of the methodology completed (skip only with explicit user direction)
• Every claim supported by at least 2 independent sources
• All sources documented with URLs and access dates
• Red flags categorized by severity (Low/Medium/High/Critical)
• Risk rating justified with specific findings
• Tools used are logged (for reproducibility)
• Report distinguishes facts from inferences
• Jurisdictions checked match the entity's actual operating countries
• Breach data timestamped (check recency)
• Report includes a limitations section (what we couldn't check and why)

Supporting Files

• references/methodology.md — Full methodology: Berkeley Protocol, Bellingcat, IntelTechniques frameworks; 8-phase company and 7-phase individual investigation sequences; 50+ red flags; professional standards
• references/tool-catalog.md — Complete 30-tool survey with descriptions, install methods, API availability, star counts, and maintenance status
• references/report-template.md — 11-section due diligence report template with guidance for each section
• references/vendor-risk-assessment.md — Full vendor cyber risk assessment template: 6-phase methodology, scoring matrix (Critical/High/Medium/Low), residual risk calculator, compliance mapping (GDPR/NIS2/DORA/PCI/HIPAA/POPIA), and recommended contract clauses
• references/fuzzy-search-strategies.md — Detailed fuzzy search techniques: name variants, company suffixes, phonetic matching, transliteration, pivot strategies, confidence tagging, reverse image search
• references/executive-brief-template.md — One-page decision-first executive brief format for C-suite audiences
• references/consultancy-model-adaptation.md — Adapted methodology for consultancy/within-environment delivery models where vendor infrastructure is out of scope for data processing
• references/jurisdiction-registry-reference.md — Quick-reference for business registries, court systems, and regulatory bodies by country
• scripts/install-tools.sh — One-shot installation script for all Python and Go tools
• scripts/generate-variants.py — Generate name variants (formal, nickname, initial, typo, phonetic) for fuzzy searching
• README.md — Project overview, quick start, and GitHub-ready documentation
• CHANGELOG.md — Version history and changes