webapp-probe

## Contents - Scope & authorization (blast-radius labels) - Passive analysis of captured traffic: 1. Missing or weak security headers 2. Insecure session cookies 3. Hardcoded secrets & API tokens in bodies 4. Improper error handling (stack traces, DB errors, debug pages) 5. Outdated / dev-mode technology fingerprints 6. RCE-prone parameters in captured requests - Active probing of the live target: 7. Disclosure-path brute (.git/config/logs, CVE paths) 8. Outdated / vulnerable software port probe 9. Wayback forgotten-endpoint mining + re-probe 10. Dependency confusion (sourcemaps -> npm) - Runnable snippets - Output - Reference files: `references/secret-regexes.md`, `references/error-signatures.md`, `references/directory-signatures.md` ## Scope & authorization Only run against a web application the user owns or is contractually engaged to test. This skill has two halves, each labelled by blast radius: - **Passive (classes 1-6)** - inspects responses and requests you already captured (proxy history, an authenticated crawl, a saved sitemap, files on disk). Sends **no new HTTP requests**, mutates nothing, and is safe against a frozen capture. - **Active (checks 7-10)** - sends live requests to the target and, for Wayback (9) and dependency confirmation (10), to third-party services (the Internet Archive, the npm registry). Check 8 is **aggressive** (multi-port sweep). Run these only in scope, and prefer a low-traffic window. Take inputs as whatever you already hold: captured responses/requests for the passive half, and the target's domains / `domain:port` pairs for the active half. Report a finding exactly where each section says "Report a finding when...". ## Passive analysis of captured traffic Each class is one pass over the capture; each yields per-target findings. ### 0. Extensive attack surface flushout and coverage Work through the given scope of the target, and work to maximize all the attack surface expansion via all techniques that comes to mind. Take a webapp for example - seeing a 404 is one thing, but subdomain enumeration expanding the targets, on each running available browser tools to get extended sitemap, available source code, backup files, flush errors and stack traces with unexpected inputs and more. Coverage is most important as you have to invent and act on lots of variations per the target type tailored to any customization opportunity you may discover that can further the search (e.g. if its next.js version xyz do we know how to trigger errors on there). ### 1. Missing or weak security headers For every captured response, case-insensitively collect the response header names and report each of the ten headers below that is absent. Report a finding when: a response is missing any of these. Remediation is "add the `<Header>` in responses returning from the server". | Header | Why it matters | |---|---| | X-Frame-Options | Blocks embedding the site in iframes (clickjacking). | | Content-Security-Policy | Controls which resources (e.g. JS) the page may load. | | Strict-Transport-Security | Forces HTTPS, including on the initial HTTP connection. | | Permissions-Policy | Disables specific browser APIs; propagates to iframes. | | X-Content-Type-Options | Stops MIME sniffing; browser trusts only the declared Content-Type. | | Referrer-Policy | Prevents leaking sensitive query params via the Referer to third parties. | | Cross-Origin-Resource-Policy | (xs-leaks) Restricts cross-origin access to the document's resources. | | Cross-Origin-Embedder-Policy | (xs-leaks) Blocks loading cross-origin resources that don't grant permission via CORP/CORS. | | Cross-Origin-Opener-Policy | (xs-leaks) Stops popup/opener documents from reaching the page's global object. | | Cache-Control | Prevents responses being cached in intermediary proxies and leaked to other users. | ### 2. Insecure session cookies Look only at cookies that look session-bearing - the cookie **name** (case-insensitive) contains `sess`, `token`, `auth`, `jwt` or `sid`. For each such `Set-Cookie` with a non-empty value, report a finding when any of: - **No attributes at all** - the cookie string has no `;` or no `=`. The whole cookie is insecure. - **Missing `HttpOnly`** - `Cookie '<name>' doesn't have the HTTPOnly attribute set`. Fix: set HttpOnly on important cookies. - **Missing `Secure`** - `Cookie '<name>' doesn't have the Secure attribute set`. Fix: set Secure. - **`SameSite=None`** - `Cookie '<name>' has the SameSite attribute set to 'none'`. Fix: set SameSite=Strict, or at least Lax. - **Long lifetime** - an `Expires` (parsed as a unix timestamp here) more than **14 days** out. `Cookie '<name>' has a long expiration date set of more than N days`. Fix: give important cookies a short-lived expiration. Cookies whose name matches none of those, and empty-valued cookies, are skipped to keep the signal high. ### 3. Hardcoded secrets & API tokens in bodies Run the secret-regex set in `references/secret-regexes.md` over every captured response body. Report an INSIGHT (not a hard finding - these are high-value leads that need confirmation) when: any pattern matches. Record `<secret name> ( <matched value(s)> ) while requesting <paths>`; de-duplicate matches per target and truncate the joined match string to 400 chars. The set (~60 patterns) covers provider credentials and identifiers: OpenAI legacy/`sk-proj-`/`sk-svcacct-` and Anthropic `sk-ant-`, npm `npm_`, Discord bot tokens & webhooks, DB connection URIs with embedded creds (MongoDB, Redis, PostgreSQL, MySQL, MSSQL), Azure Blob / GCS / Firebase Storage bucket URLs, Slack tokens/webhooks, GitHub classic/fine-grained PATs and `git+https` URIs, AWS access-key IDs / MWS tokens / S3 bucket URLs, Google API keys / OAuth / GCP service-account JSON / `ya29.` access tokens, Stripe/Square/PayPal Braintree/Picatic/Mailgun/MailChimp payment & mail keys, DigitalOcean, HuggingFace, Grafana Cloud, Segment, Okta, Fullstory, Adobe, Heroku, Telegram, Firebase URLs, and a generic `"Password": "..."` JSON pattern. Full patterns in the reference file. The reference file's **False-positive gates** section is not optional - it is what decides whether a match is worth reporting: a credential-keyword gate on the generic connection string, a placeholder/localhost filter on the connection URIs (without it those five rows are net-negative), a snowflake decode for Discord tokens, a repetition floor on the remaining unanchored rows, and the rule that rows tagged *(identifier)* are public by design and get an insight with a next step attached, never a credential finding. Secrets in third-party JS may be intentional publics (e.g. a Google Maps browser key) - verify before reporting as a leak; treat the finding as a lead. ### 4. Improper error handling - stack traces, DB errors, debug pages Run the error-signature set in `references/error-signatures.md` over every captured response body. **First skip responses that are JavaScript bundles** (they produce false positives): skip the body if it matches `sourceMappingURL|window\.addEventListener|window\.document|webpackJsonp|jQuery`. Report a finding when: any signature matches; record `<signature name> ( <match> ) while requesting <paths>`. The set (~130 signatures) spans: SQL/DB errors (MySQL, MSSQL OLE DB, PostgreSQL, Oracle `ORA-`, DB2, SQLite, MongoDB `E11000`, Elasticsearch `mapper_parsing_exception`); language runtime exceptions (Java `java.lang.*`/`java.io.*`/`java.sql.*`, Python `Traceback`/`ValueError`/`KeyError`/etc., Ruby, C++ `std::*`, .NET `System.*`); framework error pages (Django, Flask/Werkzeug, Laravel/Symfony, Spring/Spring Boot/Hibernate, Rails `ActionView::Template::Error`, Express.js, ASP.NET "Server Error in '/' Application"); **insecure-deserialization** gadget-chain traces (Java `ObjectInputStream.readObject`, `InvokerTransformer`/`LazyMap`/`ChainedTransformer`, `readObject`/`writeObject`, Python `pickle`/`yaml.load`, Ruby `Marshal.load`/`YAML.load`, PHP `unserialize(`, C# `BinaryFormatter`, Go `encoding/gob`, Jackson); config/secret disclosure (`<compilation debug="true"`, `DATABASES = {...}`, `secret_key_base`, Web.config, Node config errors); and CSRF/JWT errors. Full name+pattern table in the reference file. ### 5. Outdated / dev-mode technology fingerprints Because this is a passive body-string search (not endpoint probing), the patterns are deliberately precise. Run over every captured response body; report an INSIGHT when any matches (`<name> ( <match> ) while requesting <paths>`). This is the passive counterpart to the active port probe in check 8 - a passive hit tells you where to point check 8. | Fingerprint | Body regex | Meaning | |---|---|---| | Tomcat Dev Interface | `tomcat\.gif` | Default Tomcat page/manager exposed. | | Jenkins Dev Interface | `Welcome to Jenkins!` | Jenkins UI reachable. | | Werkzeug Dev Interface | `Werkzeug powered traceback interpreter` | Flask/Werkzeug interactive debugger (RCE-prone). | | AD Self Service Management Server | `adscsrf` | ADSelfService Plus portal. | | Frontend Code Editor | `ace_editor|codemirror|monaco-editor` | In-page code editor surface. | | Pulse Secure File Read | `dana-na\/css` | Pulse Secure (CVE-2019-11510 arbitrary file read). | Reference: nuclei-templates `http/cves/2019/CVE-2019-11510.yaml`. ### 6. RCE-prone parameters in captured requests Over captured **requests**, inspect query-string params and body params (parsed by Content-Type: XML tags, JSON keys, or `x-www-form-urlencoded` keys). Report an INSIGHT when: a parameter **key** matches the RCE-suggestive wordlist below, or a parameter **value** equals `##class` (a common template/gadget marker). Record which location matched, e.g. `Query parameter key 'cmd' was found at path '<path>'` or `JSON body parameter key 'exec' was found at path '<path>'`. These names don't prove a vuln - they flag parameters worth targeted, authorized command-injection testing. Parameter keys (case-sensitive match, as captured): ``` daemon, execute, cmd, cli, ip, xp_cmdshell, CSPCHD, exec, func, function, command, eval, shell, shell_exec, popen, proc_open, bash, python, system, payload, cmdline, exe, execcommand, exec_code, exec_cmd, executeshell, cmd_exec, cmd_inject, cmd_shell, cmd_script, run, runcmd, runcommand, shellcode, shellexec, shellcmd, command_prompt, process, terminal, execute_command, exec_file, load_module, load_script, proc_cmdline, runtime_exec, shell_execute, system_command, sys_command ``` Parameter values: `##class`. Requests with a body but no Content-Type, or an unrecognized Content-Type, are skipped. ## Active probing of the live target These send live requests. Confirm scope first; they leave logs on the target (and, for 9-10, on third parties). ### 7. Disclosure-path brute - *active* For each domain, GET a curated set of source/config/log/known-CVE paths (no redirects, TLS verification off, 10s). Each signature carries a path, an expected **status regex** and a **body regex**; a match must satisfy both. The full 30-signature table lives in `references/directory-signatures.md` (e.g. `/.git/config`, `/.git/config~`, `/.git`, `/.svn`, `/package.json`, `/jsconfig.json`, `/config.json`, `/info.php`, `/phpinfo.php`, `/web.config`, `/global.asa`, `/storage/logs/laravel.log`, `/wp-content/debug.log`, Eclipse Jetty `/WEB-INF/web.xml` and its `/%2e/`-prefixed traversal variant, Telerik `/Telerik.Web.UI.WebResource.axd?type=rau`, Pulse `/dana-na/`, `/adminer.php`, `/phinx.yml`, the `.env` family, `/.npmrc`, ten Swagger/OpenAPI doc paths, and Django-debug `/djgo` expecting a **4xx** page containing `DEBUG =`). **False-positive controls (essential - catch-all responders are common):** - Drop any response whose body contains `404 Not Found` or `This page can't be displayed`. - Hash (MD5) each matching response body; collapse duplicates so one generic page counted many times becomes one hit. - If **every** signature matches (hit count == number of signatures), discard the whole domain - the server answers 200 to everything and none of it is real. **Report a finding when**: at least one - but not all - signatures match after dedup. Severity: Information Disclosure (rate individually; exposed `.git`/`.svn` or a live source/config/log file is often high impact). ### 8. Outdated / vulnerable software port probe - *aggressive* For each `domain`, probe management interfaces of RCE-prone stacks across their typical ports, matching a substring in the response body. Cache each port's response and reuse it so overlapping port lists don't re-request. On 80/443 use plain `http://host` / `https://host`; on any other port try **both** http and https (label the hit `port (HTTP)` / `port (HTTPS)`). | Service (RCE surface) | Path | Body substring | Ports | |---|---|---|---| | Apache Tomcat | `/` | `Tomcat` | 80, 443, 8080, 8081, 8089, 8090, 9090, 9091 | | Jenkins | `/login` | `Jenkins` | 80, 443, 8080, 8081, 8089, 8090, 9090, 9091, 7443, 8443, 9443 | | Werkzeug debugger | `/console` | `Werkzeug powered traceback interpreter` | 80, 443, 8080, 4000 | | ManageEngine ADSelfService | `/showLogin.cc` | `adscsrf` | 8080, 8889 | | InterSystems Caché | `/` | `CSPCHD` | 443 | | ASP.NET ViewState | `/` | `__VIEWSTATE` | 443 | | Jolokia / Java RMI | `/jolokia` | `jolokia` | 443 | | Plone CMS | `/` | `Plone` | 443 | **Report a finding when**: any service substring appears. Severity: potential RCE insight - report `"<service> found on port(s) <list>"`. The Werkzeug `/console` interactive debugger and an exposed Jolokia agent are directly exploitable; the rest are version-review leads. ### 9. Wayback forgotten-endpoint mining + re-probe - *active-3rdparty (harvest), then active (re-probe)* **Harvest (active-3rdparty).** Query the Internet Archive CDX index per domain: ``` https://web.archive.org/cdx/search/cdx?url={domain}*&collapse=urlkey&limit=40000&fl=urlkey,timestamp,original,mimetype,statuscode,length&output=json ``` The archive rate-limits ~15 req/min and imposes a ~5-minute block; use a retry/backoff of 5m -> 10m -> 20m on HTTP 429. Drop static/noise by mimetype (`application/javascript`, `text/javascript`, `application/x-javascript`, `text/css`, `image/png`, `image/gif`, `font/woff`) and by extension in the URL (`.png .js .css .jpg .JPG .jpeg .gif .ttf .tif .eot .woff .woff2 .pdf .otf .svg .ico .html .swf .styl`). **Filter to the interesting URLs** (this de-noising is the point): - Skip URLs longer than 1000 chars. - Skip "cache-buster / hash-looking" URLs: those matching `^(?=.*[a-z])(?=.*[A-Z])(?=.*[0-9])(?!.*\?)(?!.*\=)(?!.*\&)(?!.*\_).*$` (mixed-case+digits, no query, no underscore). - Skip boring path substrings: `how-it-works, promotions, images, google-tag-manager-cart, terms-of, -products, frequently-asked-questions, wewaw, news, wp-json, wp-admin, diy`. - Require a query string (`?...`); keep only **distinct** `(path, frozenset(query-param-names))` tuples. - Compute the 20 most common path segments / query-param names across the survivors; keep an element only if it recurs **>5** times and its length is **>3 and <11**; then **drop** any URL that contains one of those common elements - i.e. keep the *unusual* endpoints, discard the site-wide boilerplate. Report the surviving URL set as an Information Disclosure insight (archived, possibly-still-live parameterized endpoints worth manual testing). **Targeted re-probe (active).** For any surviving URL whose path matches a fingerprint below, send a live GET (5s) to the target and match status + body: | Fingerprint | Path regex | Status | Body regex | Meaning |