probeagent

Understand and modify ProbeAgent's attack catalog. LOAD THIS when adding, removing, or editing an attack category or its strategies, when touching src/probeagent/attacks/, when the "N attack categories / M strategies" doc test fails, or when reasoning about profiles (quick/standard/thorough). Encodes the auto-discovery registration, the BaseAttack grading knobs, and the critical 12-vs-13 category count invariant that blocks releases.

Run the tests, lint, commit, and cut a release for ProbeAgent (the probeagent-ai package). LOAD THIS before running pytest/ruff, before any `git commit`, before bumping the version, or before publishing to PyPI. Encodes the non-obvious gotchas: the pre-commit hook hangs so commits need --no-verify, commits must carry NO AI-attribution trailer, the version lives in three files, and PyPI publishing fires off a GitHub Release and is irreversible.

Recover this repo's working copy when files read as empty/corrupt or reads hang. LOAD THIS the moment you see any of: a file that `ls`/`stat` shows as N bytes but reads as empty; md5 == d41d8cd98f00b204e9800998ecf8427e on a non-empty file; "short read: Undefined error: 0" from git/grep; ImportError "No module named 'probeagent'" even though tests pass; pytest failing on a symbol that exists in source; a Read/cat/open that hangs forever. This machine's filesystem intermittently wedges — inode metadata stays intact while data blocks go empty or unreadable. This is the single most destructive trap in this project; a model that doesn't recognize it will "fix" phantom bugs for hours.

Run the "same agent, different backing models" resistance comparison and build the leaderboard. LOAD THIS when asked to compare models, stress-test an agent across models, produce the bakeoff report, or work with tools/run_bakeoff.sh, tools/model_bakeoff.py, tools/leaky_agent.py, or the --model flag. Encodes the OpenRouter setup, the secret-key hygiene, the wedged-fs invocation, and cost control.

Follow ProbeAgent's house rules for files, licensing, packaging, changelog, and what belongs in the repo. LOAD THIS when creating a new source file, editing packaging (pyproject), writing changelog entries, or deciding whether something (esp. marketing) belongs in the repo at all. Prevents the common mistakes that get caught in review.

Work with ProbeAgent's target adapters (http, openclaw, mock, mcp) and write/​fix its tests. LOAD THIS when touching src/probeagent/targets/, adding target-facing tests, mocking HTTP with respx, testing the async MCP path, or debugging a test that passes alone but fails in the full suite. Encodes the mock schemes, the respx pattern, and the SeedCorpusAttack.STRATEGIES global-mutation trap that silently breaks unrelated tests.

Reason about ProbeAgent's grading (Compromised/Resisted/Blocked) and its JSON report shape. LOAD THIS when interpreting scan results, changing the scorer/reporter, writing anything user-facing about verdicts, consuming the JSON programmatically, or touching docs/demo copy. Encodes the three-verdict model, the retired-vocabulary guardrail, the exact JSON schema, and the --output json vs -f gotcha.