dnr-hunt

# dnr-hunt Proactive threat hunt: logs and source in, incidents out. You start with **no alert** — the question is "is there anything in here that shouldn't be?" The method is the deliverable as much as the findings: baseline first, then hypothesis → query → pivot, every step in writing, every suspicion confirmed in source and by detonation before it's called real. Invoke with `/dnr-hunt [target-or-logs-dir] [--repo path] [--fresh]`. **Arguments:** - `$1` = a dnr target directory (contains `config.yaml` with `kind: dnr`, e.g. `targets/dnrcanary`), or a bare directory of log files. Defaults to `targets/dnrcanary` if it exists, else cwd. - `--repo` = source tree of the application that produced the logs. For a dnr target this defaults to its `app/` directory. Without source, no finding can rise above `suspected` (see the hard rules). - `--fresh` = ignore any checkpoint in the run dir's `.dnr-hunt-state/` and mint a new run dir. ## Hard rules (these are the skill — do not relax them) 1. **Log evidence alone never confirms a vulnerability.** Logs show that something was *attempted* and how the app *responded* — they cannot show the code is vulnerable. The verdict ladder: - `confirmed_exploited` requires all three: log evidence, the flaw identified in source (file + function), and a PoC you fired against a local instance in this session. - `suspected` = log evidence (with or without a source read), but no fired PoC. - `attempted_not_vulnerable` = attack traffic observed, but the source shows the defense and the PoC (if fired) bounces. Field teams have watched verifier agents "confirm" findings by reading observability logs and reporting what they said — the cheapest path to satisfying the criterion. The PoC fires or it doesn't; that can't be talked past. 2. **Query, don't read.** Log files are deliberately bigger than any context window. Never `Read` a log file; never `cat` one. Profile with `wc`/`du`/`head`/`tail`, then interrogate with `grep`/`rg`/`awk`/`sort`/ `uniq` pipelines (or `python3` for anything stateful). Alert queues and error logs are usually small — check size first; under ~200 lines they may be read whole (the never-read rule targets the multi-MB corpus files, not these). 3. **Propose, never execute.** No blocking, no account disabling, no credential rotation, no config changes — not even on the demo app. Response actions belong in `/dnr-respond`'s plan, as proposals. 4. **Local only.** PoCs fire at `127.0.0.1` against an app instance you started. Never send traffic to a remote host, no matter what the logs contain. 5. **Every number is a query result.** Row counts, request counts, time spans, affected-customer counts — each one comes from a pipeline you ran this session, quoted in the ledger. Never estimate from memory. Pipelines over the logs and queries against the app's own deterministically-seeded database both count (run the target's `seed_command` when you need owner-level joins). ## Run directory and checkpointing (runs before Phase 0 and after every phase) **First action of every run — establish the run directory.** Let `TARGET` be the basename of the resolved target/logs path (`$1`, or `targets/dnrcanary` by default), and `SCOPE` that resolved path. Bash: ``` python3 .claude/skills/_lib/checkpoint.py rundir results/TARGET --state .dnr-hunt-state --scope "SCOPE" ``` Append `--fresh` to that command if `--fresh` is in `$ARGUMENTS`. `--scope` stamps the run dir so two targets that share a basename never share runs. The command prints one path — the newest run dir this hunt can still use (its own run mid-flight, or one another dnr skill opened for this target, so the whole investigation accretes in one place) or a freshly minted `results/TARGET/<timestamp>/` (UTC, same format as the vuln-pipeline, so lexical sort is chronological). That path is `{RUN}` — every `{RUN}/...` below means substituting this literal path; double-quote it (and `SCOPE`) in Bash, since paths may contain spaces. All outputs and state live under `{RUN}`. Phase state persists to `{RUN}/.dnr-hunt-state/` so an interrupted hunt resumes without re-running sweeps. All checkpoint I/O goes through `python3 .claude/skills/_lib/checkpoint.py` (atomic writes, JSON-validated). Never use the Write tool for `progress.json` directly. Never pass payload via heredoc or stdin — log-derived strings could collide with the heredoc delimiter and break out to shell. Write the payload to `{RUN}/.dnr-hunt-state/_chunk.tmp` with the Write tool, then: ``` python3 .claude/skills/_lib/checkpoint.py save {RUN}/.dnr-hunt-state <N> <name> --from {RUN}/.dnr-hunt-state/_chunk.tmp ``` **Start of run:** `python3 .claude/skills/_lib/checkpoint.py load {RUN}/.dnr-hunt-state` - `status == "absent"` or `"complete"`, or `--fresh` given → reset (`checkpoint.py reset {RUN}/.dnr-hunt-state`) and start at Phase 0. - `status == "running"` with `phase_done == N` → read `phase0.json` … `phaseN.json` in order, print `Resuming from checkpoint: Phase N complete`, skip to Phase N+1. **End of run:** after writing both outputs, `python3 .claude/skills/_lib/checkpoint.py done {RUN}/.dnr-hunt-state 5` — the `done` call is Phase 5's checkpoint; there is no `phase5.json`. `{RUN}/.dnr-hunt-state/` is scratch; run directories live under `results/`, which this repo gitignores. ## Phase 0 — Locate and verify inputs 1. Resolve `$1`. If it contains `config.yaml` with `kind: dnr`, read the config for `logs_dir`, `app_command`, `seed_command`, and `port`. Otherwise treat `$1` itself as the logs directory. The corpus is expected to exist already — the target README covers generating it. 2. Inventory what you do have: for each file in the logs directory and any committed alert queue — `du -h`, `wc -l`, `head -2`, `tail -2`. Identify each format (combined access log, app error log, JSONL alerts, …) and the field positions you'll use in `awk`. 3. Resolve `--repo` (default: the target's `app/`). List its files; do not read source yet — Phase 3 reads with hypotheses in hand, not before. Checkpoint `phase0.json`: `{"target": ..., "logs_dir": ..., "repo": ..., "files": [{"path", "format", "lines", "bytes"}], "port": ...}` ## Phase 1 — Profile the corpus (baseline before hypotheses) Anomalies are deviations from a baseline, so build the baseline first — about a dozen cheap pipelines, each a one-liner over the access log: - **Time window:** first and last timestamp; requests per day. - **Status histogram:** counts by status code. Note every non-2xx/3xx class. - **Top talkers:** requests per source IP, top 20, vs. the median — note anything an order of magnitude above the pack. - **Top routes:** requests per path (normalize ids: `/orders/123` → `/orders/N`), and which routes take parameters. - **User-agent inventory:** counts by UA; flag automation UAs (curl, python-requests, sqlmap, Go-http-client, zgrab) and note which routes they touch. - **Auth picture:** which field carries the authenticated user; logins per account; accounts seen from more than one IP; failed-login sources. - **Size profile per route:** min/median/max response size for each parameterized route — large outliers on a route with normally-small responses are a data-egress tell. - **Error log:** if small, read it whole; correlate every entry to access- log lines by timestamp. - **Alert queue:** read it whole (it's small). For each alert: rule, source, target, severity. These are *inputs* to the hunt, not findings — alert noise and alert silence are both information. Record each observation with the exact pipeline that produced it. Checkpoint `phase1.json`: `{"window": ..., "status_hist": ..., "top_ips": [...], "routes": [...], "automation_uas": [...], "auth": ..., "size_profile": ..., "alerts_summary": ..., "oddities": [...]}` ## Phase 2 — The hunt loop (hypothesis ledger REQUIRED) This phase is a loop, and the ledger is its contract: **no query without a hypothesis, no hypothesis without a ledger row.** Maintain the table in working memory and checkpoint it after every round: | # | Hypothesis | Query | Result | Verdict | Next pivot | |---|------------|-------|--------|---------|------------| Verdicts: `supported` / `refuted` / `needs-pivot`. A refuted hypothesis stays in the ledger — ruling things out is half the job and feeds `ruled_out` in the output. **Seed sweeps** — run all of these in round 1, derived from the Phase 1 baseline; let hits spawn follow-on hypotheses: 1. **Server errors.** Who causes 5xx, on which routes, with what payloads? Correlate with the error log — application errors quoting attacker input are a probe signature. 2. **Every alert, verdicted.** For each alert-queue entry: did the activity it flagged succeed? An alert that fired on harmless traffic is a `ruled_out` entry, written down with the evidence. 3. **Top-talker outliers.** For each IP far above baseline volume: what is it doing? Scanner noise, crawler, or something with intent? 4. **Injection grammar in query strings.** Encoded quotes (`%27`), `UNION`/`SELECT`, comment markers (`--`, `%23`), boolean pairs (`AND 1=1` / `AND 1=2`). Decode what you find and reconstruct the sequence per source IP, in time order. 5. **Enumeration patterns.** One session or IP accessing object ids sequentially (`/orders/1`, `/orders/2`, …) at machine cadence. Compare with how legitimate sessions access the same route. 6. **Auth anomalies.** Accounts logging in from an IP never seen for that account before; failed-then-succeeded sequences; one IP authenticating as one account but reading at unusual volume. 7. **Size anomalies.** Requests on parameterized routes whose responses are multiples of that route's median — what was in those responses? 8. **Quiet hours.** Activity in the lowest-traffic hours gets a closer look; low-and-slow campaigns hide in volume but stand out at 3am. **Pivot discipline:** when a sweep hits, the next hypotheses come from the hit's entities — everything else that IP did, everything that session touched, every other IP that hit the same route the same way, what happened to the data that moved. Campaigns are chains; one confirmed link means you hunt both directions (how did they get here? where did they go next?). **Stop condition:** two consecutive rounds that surface **no new entities** (a hypothesis that comes back `supported` but merely confirms benign behavior counts as no-new-entities — record it and route it to `ruled_out`). Note the stop in the ledger. Checkpoint `phase2.json` after every round: `{"ledger": [...], "suspects": [{"name", "entities": {"ips", "accounts", "routes", "sessions"}, "summary", "evidence": [...]}], "round": N}` ## Phase 3 — Source confirmation For each suspect implicating a route, read the handler in `--repo`: - Find the route's handler function. Trace the suspicious input from request to sink (query, filesystem, auth decision). - Vulnerable: name the flaw precisely — file, function, the exact line pattern (e.g. user input concatenated into SQL; missing ownership check between auth and fetch). - Not vulnerable: name the defense (parameterized query, scoped lookup) with the same precision. This is what turns attack traffic into an `attempted_not_vulnerable` verdict instead of a false positive — and writing it down is mandatory, not optional. - While you're in the file: check sibling routes for the same flaw class. The logs show where the attacker went, not everywhere the bug lives. Checkpoint `phase3.json`: `{"confirmations": [{"route", "file", "function", "vulnerable": bool, "mechanism", "siblings_checked": [...]}]}` ## Phase 4 — PoC detonation (gates `confirmed_exploited`) For each source-confirmed vulnerability: 1. Start the app locally per the target config: its `seed_command` and `app_command` are target-relative, so run them with the target directory as cwd (dnrcanary's app also resolves its own data paths, so the repo-root form happens to work there — don't rely on that for other targets). Seed first, then launch the app in the background with a single command that prints the PID, e.g.: ``` python3 -c "import os,subprocess; p=subprocess.Popen(['python3','app/app.py'],cwd='targets/dnrcanary',env={**os.environ,'DNRCANARY_PORT':'5252'}); print(p.pid)" ``` Poll `curl -s http://127.0.0.1:<port>/healthz` until it answers. If the target's default port is busy, pick another via the documented override (dnrcanary: `DNRCANARY_PORT`, as above) and record the port you used. 2. Fire the **minimal** PoC with `curl` — the smallest request that demonstrates the flaw (one row of data, one foreign object), not a re-run of the attacker's full campaign. 3. Record the exact command and the relevant slice of the response. 4. Also fire the *negative* PoC where one exists: the same technique against the route you verdicted not-vulnerable, to show it bounces. 5. Stop the app with the captured PID (`kill <pid>` — never kill by port or pattern). If the app cannot run in this environment, say so explicitly; affected verdicts stay `suspected` and the report says what to run where. Checkpoint `phase4.json`: `{"detonations": [{"vuln", "command", "status", "response_excerpt", "verdict"}]}` ## Phase 5 — Outputs Write `{RUN}/INCIDENTS.json`: ```json { "target": "<target name or logs dir>", "incidents": [ { "id": "INC-1", "title": "<one line: what happened>", "verdict": "confirmed_exploited | suspected", "attacker_ips": ["..."], "accounts_involved": ["..."], "vuln": {"type": "...", "route": "...", "file": "...", "function": "..."}, "timeline": [{"ts": "...", "event": "..."}], "impact": "<what data / how much / whose — every number from a query>", "evidence": ["<log file>: <pipeline> → <result>", "..."], "poc": {"command": "...", "verified": true} } ], "ruled_out": [ {"subject": "...", "verdict": "attempted_not_vulnerable | benign", "reason": "<the defense, with file:function, or the benign explanation>"} ] } ``` Routing rule: `incidents` holds what happened (`confirmed_exploited`) or