analyzing-web-server-logs-for-intrusion

name	analyzing-web-server-logs-for-intrusion
description	Parse Apache and Nginx access logs to detect SQL injection attempts, local file inclusion, directory traversal, web scanner fingerprints, and brute-force patterns. Uses regex-based pattern matching against OWASP attack signatures, GeoIP enrichment for source attribution, and statistical anomaly detection for request frequency and response size outliers.
domain	cybersecurity
subdomain	security-operations
tags	["analyzing","web","server","logs"]
version	1.0
author	mahipal
license	Apache-2.0
nist_csf	["DE.CM-01","RS.MA-01","GV.OV-01","DE.AE-02"]

Analyzing Web Server Logs for Intrusion

When to Use

• When investigating security incidents that require analyzing web server logs for intrusion
• When building detection rules or threat hunting queries for this domain
• When SOC analysts need structured procedures for this analysis type
• When validating security monitoring coverage for related attack techniques

Detection Gaps & Validation

• Regex misses encoded SQLi: UNION SELECT and OR 1=1 are easy, but URL-encoding (%55NION), inline comments (UNI/**/ON SE/**/LECT), case mixing, and double-encoding bypass naive signatures. Normalize (URL-decode twice, strip comments, lowercase) before matching, and prefer libinjection-style tokenization over literal strings.
• POST bodies and headers are invisible in access logs: Apache/Nginx access logs record only the request line and query string — SQLi/XSS/LFI in POST bodies, cookies, or custom headers won't appear unless mod_security audit logging or body capture is enabled. State this coverage gap explicitly; an empty result over access.log is not proof of no attack.
• Scanner UA spoofing: sophisticated actors set a Chrome User-Agent, so nikto|sqlmap|gobuster UA matching only catches lazy scans. Add request-rate, 404-ratio, and path-fuzzing-burst heuristics independent of UA.
• LFI/traversal evasion: beyond ../etc/passwd, test ..%2f, ..%252f (double-encode), ....//, and absolute /etc/passwd with no traversal — match root:.*:0:0: in responses, not just the request.
• Validate the rules fire: replay a labeled attack log (sqlmap run, dirbuster sweep, a brute-force burst >50 POSTs/5min to /login) through the parser and confirm each rule and the GeoIP enrichment populate. FP tuning: WAF health checks, search-engine bots, and legitimate apps passing SQL keywords in search fields cause false hits — baseline trusted IPs/paths.

Prerequisites

• Familiarity with security operations concepts and tools
• Access to a test or lab environment for safe execution
• Python 3.8+ with required dependencies installed
• Appropriate authorization for any testing activities

Instructions

1. Install dependencies: pip install geoip2 user-agents