| name | performing-paste-site-monitoring-for-credentials |
| description | Monitor paste sites like Pastebin and GitHub Gists for leaked credentials, API keys, and sensitive data dumps using automated scraping and keyword matching to detect breaches early. |
| domain | cybersecurity |
| subdomain | threat-intelligence |
| tags | ["paste-monitoring","credential-leak","pastebin","data-breach","threat-intelligence","osint","early-warning"] |
| version | 1.0 |
| author | mahipal |
| license | Apache-2.0 |
Performing Paste Site Monitoring for Credentials
Overview
Paste sites (Pastebin, GitHub Gists, Ghostbin, Dpaste, Hastebin) are frequently used as staging areas for leaked credentials, database dumps, API keys, and sensitive data before wider distribution on dark web forums and Telegram channels. Monitoring these sites provides early breach detection, enabling organizations to respond before stolen data is weaponized. This skill covers building automated paste site monitors using the Pastebin Scraping API, keyword-based alerting, credential pattern matching, and integration with incident response workflows.
Prerequisites
- Python 3.9+ with
requests, beautifulsoup4, regex, pymisp libraries
- Pastebin PRO account with Scraping API access ($49.95/month for programmatic access)
- GitHub API token for Gist monitoring
- Keyword lists specific to your organization (domains, project names, internal terms)
- Elasticsearch or database for paste storage and search
Key Concepts
Paste Site Threat Landscape
Over 300,000 user credentials are posted on Pastebin annually, averaging 1,000 username/password pairs per leak. Paste sites serve three primary threat intelligence purposes: early breach detection (credentials appear on paste sites before dark web), threat actor profiling (actors use paste sites for C2 configuration, data staging, tool sharing), and malware discovery (encoded payloads, configuration files, C2 addresses).
Monitoring Approaches
Active monitoring queries paste site APIs or scraping endpoints at regular intervals. The Pastebin Scraping API provides real-time access to new public pastes. For GitHub, the search API allows monitoring Gists and repository commits for exposed secrets. Passive monitoring uses services like IntelX, Dehashed, or Have I Been Pwned that aggregate paste site data.
Credential Pattern Detection
Effective monitoring uses regex patterns for email:password combinations, API keys (AWS, Azure, GCP, Stripe, Twilio), database connection strings, private keys (SSH, PGP), JWT tokens, and internal hostnames/URLs. Organization-specific keywords (domain names, product names, employee names) reduce false positives.
Practical Steps
Step 1: Pastebin Scraping API Monitor
import requests
import re
import json
import time
from datetime import datetime
class PastebinMonitor:
SCRAPING_URL = "https://scrape.pastebin.com/api_scraping.php"
RAW_URL = "https://scrape.pastebin.com/api_scrape_item.php"
def __init__(self, keywords, output_dir="paste_alerts"):
self.keywords = [k.lower() for k in keywords]
self.output_dir = output_dir
self.seen_keys = set()
self.credential_patterns = {
"email_password": re.compile(
r'[\w.+-]+@[\w-]+\.[\w.]+[\s:;|,]+[\S]{6,}', re.IGNORECASE),
"aws_key": re.compile(
r'AKIA[0-9A-Z]{16}'),
"aws_secret": re.compile(
r'[0-9a-zA-Z/+=]{40}'),
"github_token": re.compile(
r'ghp_[0-9a-zA-Z]{36}'),
"slack_token": re.compile(
r'xox[baprs]-[0-9a-zA-Z-]+'),
"private_key": re.compile(
r'-----BEGIN (?:RSA |EC |DSA )?PRIVATE KEY-----'),
"jwt_token": re.compile(
r'eyJ[A-Za-z0-9-_]+\.eyJ[A-Za-z0-9-_]+\.[A-Za-z0-9-_]+'),
: re.(
),
: re.(
,
re.IGNORECASE),
}
():
params = {: limit}
:
resp = requests.get(.SCRAPING_URL, params=params, timeout=)
resp.status_code == :
pastes = resp.json()
()
pastes
:
()
[]
Exception e:
()
[]
():
params = {: paste_key}
:
resp = requests.get(.RAW_URL, params=params, timeout=)
resp.status_code == :
resp.text
Exception:
():
findings = {
: [],
: {},
: ,
}
content_lower = content.lower()
keyword .keywords:
keyword content_lower:
count = content_lower.count(keyword)
findings[].append({
: keyword,
: count,
})
pattern_name, pattern .credential_patterns.items():
matches = pattern.findall(content)
matches:
findings[][pattern_name] = {
: (matches),
: matches[:],
}
cred_count = (
m[] m findings[].values()
)
findings[] cred_count > :
findings[] =
findings[]:
findings[] =
cred_count > :
findings[] =
cred_count > :
findings[] =
findings
():
count =
iterations count < iterations:
pastes = .fetch_recent_pastes()
alerts = []
paste pastes:
paste_key = paste.get(, )
paste_key .seen_keys:
.seen_keys.add(paste_key)
content = .get_paste_content(paste_key)
content:
findings = .analyze_paste(content, paste)
findings[] != :
alert = {
: paste_key,
: paste.get(, ),
: paste.get(, ),
: paste.get(, ),
: paste.get(, ),
: ,
: findings,
: datetime.now().isoformat(),
}
alerts.append(alert)
(
)
alerts:
._save_alerts(alerts)
count +=
iterations count < iterations:
time.sleep(interval)
alerts
():
filename =
os
os.makedirs(.output_dir, exist_ok=)
(filename, ) f:
json.dump(alerts, f, indent=)
()
monitor = PastebinMonitor(
keywords=[, , ],
)
alerts = monitor.monitor_loop(interval=, iterations=)
Step 2: GitHub Gist and Code Search Monitoring
class GitHubSecretMonitor:
def __init__(self, github_token, org_keywords):
self.token = github_token
self.keywords = org_keywords
self.headers = {
"Authorization": f"token {github_token}",
"Accept": "application/vnd.github.v3+json",
}
def search_code(self, query, per_page=30):
"""Search GitHub code for leaked secrets."""
url = "https://api.github.com/search/code"
params = {"q": query, "per_page": per_page}
resp = requests.get(url, headers=self.headers, params=params)
if resp.status_code == 200:
results = resp.json().get("items", [])
print(f"[+] GitHub code search: {len(results)} results for '{query}'")
return results
return []
def search_gists(self, keyword):
"""Search public Gists for sensitive data."""
url = "https://api.github.com/gists/public"
params = {"per_page": 100}
resp = requests.get(url, headers=self.headers, params=params)
matches = []
if resp.status_code == 200:
gists = resp.json()
gist gists:
description = (gist.get() ).lower()
files = gist.get(, {})
filename, file_info files.items():
keyword.lower() description keyword.lower() filename.lower():
matches.append({
: gist[],
: gist.get(, ),
: filename,
: gist[],
: gist[],
})
matches
():
queries = [
,
,
,
,
,
]
all_findings = []
query queries:
results = .search_code(query)
result results:
all_findings.append({
: query,
: result.get(, {}).get(, ),
: result.get(, ),
: result.get(, ),
: result.get(, ),
})
time.sleep()
all_findings
gh_monitor = GitHubSecretMonitor(, [])
findings = gh_monitor.monitor_org_secrets()
Step 3: Alert and Incident Response Integration
def generate_credential_leak_alert(alert_data):
"""Generate incident alert for credential leak detection."""
alert = {
"title": f"Credential Leak Detected - {alert_data.get('severity', 'unknown').upper()}",
"source": alert_data.get("url", ""),
"detected_at": alert_data.get("detected_at", ""),
"severity": alert_data.get("severity", "medium"),
"summary": f"Paste containing organization keywords and credentials found",
"keyword_matches": alert_data.get("findings", {}).get("keyword_matches", []),
"credential_types": list(alert_data.get("findings", {}).get("credential_matches", {}).keys()),
"recommended_actions": [
"Verify if leaked credentials are valid",
"Force password reset for affected accounts",
"Rotate exposed API keys and tokens",
"Check access logs for unauthorized usage",
"Report paste for takedown",
"Update monitoring keywords if new patterns found",
],
}
return alert
Validation Criteria
- Pastebin Scraping API queried successfully with rate limiting
- Credential patterns detected (email:password, API keys, private keys)
- Organization-specific keywords matched with context
- GitHub code search identifies exposed secrets
- Alerts generated with severity classification
- Integration with incident response workflow
References
