| name | deployment-audit |
| description | Authorized deployment readiness and go-live security audit (penetration testing, pentest, security audit, verified Python exploits) for software projects, running apps, and CTOX-created Business OS apps. Use as the final app go-live gate to assess release readiness, validate scanner and browser evidence, verify authorization boundaries, demonstrate release blockers with executed Python proofs, and produce auditable reports through one Rust CLI for Pi Code and CTOX. |
Deployment Audit
Run authorized go-live security checks through one evidence-preserving CLI.
Call validated security issues release blockers. Do not describe this skill
or its commands as a general exploitation facility.
Non-Negotiable Contract
- Test only initialized scope. Never expand a URL, host, path, account, tenant,
or active technique without explicit authorization.
- A full audit is an executed workflow, not a connectivity check or scanner
summary.
- Scanner output creates candidates or observations, never findings by itself.
- A finding requires independently executed evidence and a reusable
reproduce.py bundle.
- Missing tools and unexecuted work are coverage gaps. Never invent output or
silently replace a scanner.
- A timed-out, output-limited, disk-quota, structurally invalid, or unverified
scanner run is blocked coverage. Never parse truncated/missing output into
findings or treat killed or version-drifted processes as completed.
- Do not persist credentials, cookies, bearer tokens, browser storage, raw
browser streams, or sensitive screenshots in audit state.
- Do not claim completion unless the final review JSON has
.completion_review.closable == true and .completion_review.blocker_count == 0.
Pin One State Directory
Choose the namespace once:
- Inside CTOX:
ctox appsec
- Pi Code or standalone:
pentest
For standalone/Pi runs, put global paths before the subcommand and repeat
them on every call:
pentest --state-dir /absolute/audit-state \
--tools-root /absolute/ctox-runtime/tools/appsec init --url https://app.example --json
pentest --state-dir /absolute/audit-state \
--tools-root /absolute/ctox-runtime/tools/appsec scope check --url https://app.example --json
Never rely on the current directory's implicit .pentest when the caller
provided a state directory. Do not mix implicit and explicit state in one run.
All machine-consumed calls use --json.
If the standalone binary is not installed, use:
cargo run --manifest-path src/tools/appsec-pentest/Cargo.toml -- <global-options> <command>
Select The Workflow
CTOX-Created Business OS App
Use the existing native validator and browser runners first, writing their
JSON/screenshots into the explicit audit state:
ctox business-os app validate <module-id> --installed --json
ctox business-os app smoke <module-id> --installed --json \
--output <state>/business-os-smoke.json \
--screenshot <state>/business-os-smoke.png
ctox business-os app e2e <module-id> --installed --json \
--output <state>/business-os-e2e.json \
--screenshot <state>/business-os-e2e.png
Then run the source-aware deployment audit against
<ctox-root>/runtime/business-os/installed-modules/<module-id> and the
separately scoped Business OS origin when active runtime testing is authorized.
Never pass the shell's #<module-id> fragment to an HTTP scanner: fragments
are browser-local routing, not a server target. A release claim requires the
validator, smoke, a product-specific E2E scenario, applicable tools capabilities, and the same blocker-free completion review as every other app.
Real External Audit
Use for a deployed URL without source access. Default entry is the one-shot
orchestrator audit run:
pentest --state-dir <state> --tools-root <tools> init --url <url> --json
pentest --state-dir <state> --tools-root <tools> audit run --url <url> --json
audit run requires initialized scope and chains assess, analysis,
auto-investigation of every release-blocker candidate, finding verification,
and verified exploit export (<state>/exploits/ with index.json +
index.md) in one command. Active validation stays gated: pass --active
only with a granted durable approval created up front:
pentest --state-dir <state> approval request --url <url> \
--tool nuclei --tool dalfox --allow-destructive --reason <roe-justification> --json
pentest --state-dir <state> approval grant --id <approval-id> \
--approver <operator> --high-impact-ack --reason <text> --json
pentest --state-dir <state> audit run --url <url> --active --approval-id <approval-id> --json
After audit run, the mandatory After audit run section
applies.
Granular control (same chain, step by step, e.g. for toolchain proof first):
pentest --state-dir <state> --tools-root <tools> init --url <url> --json
pentest --state-dir <state> --tools-root <tools> scope check --url <url> --json
pentest --state-dir <state> --tools-root <tools> tools capabilities --json
pentest --state-dir <state> --tools-root <tools> tools proof-run --profile full --probe-versions --json
pentest --state-dir <state> --tools-root <tools> tools readiness --profile standard --probe-versions --smoke --json
pentest --state-dir <state> --tools-root <tools> assess --profile full --url <url> --json
Before claiming support for an application family, inspect tools capabilities. Every target-relevant row must be status:implemented and have
an executable pipeline plus retained functional proof. partial and missing
are coverage blockers; prose or manual-validation cannot promote them.
Scanners carry an operational tier (core, extended, specialized). On a
fresh operator machine, install the core tier and the pinned data bundles
before expecting readiness:
pentest --state-dir <state> --tools-root <tools> tools bootstrap --check --tier core --json
pentest --state-dir <state> --tools-root <tools> tools bootstrap --execute --tier core --json
pentest --state-dir <state> --tools-root <tools> tools data --profile full --execute --json
tools bootstrap --execute brew-installs missing tier tools on macOS and
persists each install as a run record. tools data materializes the pinned
data bundles (nuclei templates, SecLists wordlist, Trivy offline DB, pinned
Semgrep rules, the Retire.js vulnerability repository, and the pinned signed
ClamAV CVD snapshot); nuclei
template-bundle and -dast runs fail closed until the bundle exists.
For mapped native Homebrew scanners, inventory verifies the exact keg version,
binary digest, install receipt, and optional SPDX SBOM automatically; do not
invent an operator identity merely to attest a package-manager installation.
Managed Python venvs are ABI/stdlib checked and repaired when broken. Networked
Go/Python/Node installer steps use bounded transient retries; Go source builds
retain at most 8 GiB of failed-download cache so the next reviewed attempt can
resume, while successful batches clean the cache.
Python-backed scanners additionally require the install plan's
.python_runtime.ok to be true after a disposable real-venv smoke proves
ensurepip, pip, pyexpat, ssl, and venv; a base-interpreter import or
an unverified python3 PATH fallback is not sufficient.
Standard/full smoke proofs also execute local positive and negative fixtures
for HTTPX, Katana, Nuclei, FFUF, dirsearch, Gospider, Semgrep multi-language
taint flows, Gitleaks, TruffleHog, Bandit, Trivy, Retire.js, Syft, ESLint,
ast-grep, ClamAV, Ndiff, and Schemathesis OpenAPI and GraphQL response
contracts. Standard and Full must retain functional fixtures for more tools
than version-only probes. The Full proof also executes retained
WebSocket, SSE, HTTP-control, authenticated transport, gRPC, identity/session,
OCI-layout, cloud/Kubernetes, and browser-runtime positive/negative fixtures.
Mobile and desktop parsers/signature contracts use crate-level positive and
negative regression fixtures; a real artifact run still requires the
host-platform verifier and scanner evidence listed below. A version banner
alone is not functional proof.
tools doctor runs exact version probes and the applicable functional smokes
by default; --skip-version-probes and --skip-smoke are explicit diagnostic
opt-outs and are not release evidence.
Production scanner runs retain sandbox-policy.json and
resource-policy.json. On macOS, filesystem/network sandboxing plus aggregate
process-group memory/process monitoring is enforced; on Linux, Bubblewrap is
mandatory and a missing backend fails closed. Windows currently supplies Job
Object resource/process-tree control but not filesystem, network, or identity
confinement; tools requirements therefore retains requirement 14 as partial
and a Windows scanner run is not sandbox-complete evidence. Treat an
unenforced sandbox backend, resource-limit termination, or secret-bearing argv
rejection as a coverage blocker, not a negative scan result.
RLIMIT_FSIZE is a pre-write single-file bound, while the aggregate 256 MiB
run-directory control is still a periodic watchdog. Until an atomic
quota-backed writable filesystem is available on every runner, requirement 17
remains partial; do not describe it as a strict aggregate pre-write quota.
For interchange, import external SARIF 2.1.0 only as unvalidated candidates
with scanner import-sarif --file <report.sarif> and export with report --format sarif. The required Full-profile taint path is the pinned Semgrep OSS
engine plus exact-commit local multi-language taint rules. CodeQL remains an
explicitly selectable extension only after independent operator review of its
license terms; when used, it consumes a prepared database and retains SARIF
code-flow evidence.
Use the typed application-family runners instead of assembling raw scanner
arguments for specialized targets:
pentest --state-dir <state> scan api --schema <openapi> --url <base-url> --authz-matrix <matrix> --active --confirm-active --approval-id <id> --json
pentest --state-dir <state> scan graphql --url <endpoint> --schema <schema> --authz-matrix <matrix> --active --confirm-active --approval-id <id> --json
ctox appsec --state-dir <state> scan grpc --host <authority:port> --protoset <file> --method <package.Service/Method> --stream-method <package.Service/Stream> --authz-matrix <matrix> --credential-ref ctox-secret://<scope>/<name> --active --confirm-active --approval-id <id> --json
ctox appsec --state-dir <state> scan transport --kind websocket|sse --url <url> --origin <origin> --authz-matrix <matrix> --credential-ref ctox-secret://<scope>/<name> --active --confirm-active --approval-id <id> --json
pentest --state-dir <state> scan identity --issuer <https-issuer> <authorization-code-options> --active --confirm-active --approval-id <id> --json
ctox appsec --state-dir <state> scan browser-runtime --target <source-root> --url <application-url> --source-id <web-stack-source> --credential-ref ctox-secret://<scope>/<name> --verify-selector <selector> --active --confirm-active --approval-id <id> --json
ctox appsec --state-dir <state> scan oci --image <registry/repo@sha256:digest> <signature-and-attestation-policy> [--registry-credential-ref ctox-secret://<scope>/<name>] --active --confirm-active --approval-id <id> --json
ctox appsec --state-dir <state> scan cloud --provider aws|azure|gcp|kubernetes|iac --scope-id <scope> --privilege-graph <graph> <provider-options> --active --confirm-active --approval-id <id> --json
ctox appsec --state-dir <state> logic contract --file <logic-contract.json> [--credential-ref ctox-secret://<scope>/<name>] --active --confirm-active --confirm-non-get --approval-id <id> --json
pentest --state-dir <state> scan composition --target <root> --kind auto|dockerfile|kubernetes|helm-rendered|helm-chart --json
pentest --state-dir <state> scan attack-surface --domain <root-domain> --inventory <redacted-inventory.json> --json
pentest --state-dir <state> scan artifact --target <package-or-binary> --type mobile|desktop|native|installer --json
Consult tools capabilities before selecting these commands. Complete API,
GraphQL, gRPC, transport, cloud, browser-runtime, and product-invariant claims
require their target-bound redacted authorization/privilege/logic evidence,
not only a successful scanner exit. Native ctox appsec resolves
ctox-secret:// references in-process; standalone pentest must fail closed
when a requested credential path cannot be resolved without placing a secret
in argv.
Artifact completion requires a real host-platform signature verifier, bounded
package inspection, ClamAV, offline Trivy, and a CycloneDX Syft inventory.
Android and iOS require decoded release manifests and verified APK/AAB/IPA
signatures; APPX/MSIX require decoded restricted capabilities and signtool;
Electron/ASAR and Tauri require normalized application-security controls;
native ELF/PE/Mach-O binaries require normative hardening verdicts. Safe,
bounded ephemeral decoding is permitted where the verifier or parser contract
requires it, but archive signature-file presence never substitutes for
verification and decoded bodies are never retained.
OCI completion requires immutable digests plus structured Trivy, CycloneDX
Syft, Cosign signature, and every requested attestation result. Helm charts are
rendered only through the checksum-pinned managed Helm binary and immediately
scanned as bounded rendered manifests. Browser runtime completion requires all
four normalized controls (dom_xss, service_worker, postmessage, and
browser_storage) to execute in an authenticated isolated session; static
source evidence alone remains incomplete. Load references/workflow.md for
the exact evidence schemas and native pipeline recipes before executing these
modes.
Add --active only when the rules of engagement authorize the planned active
stages. If login or account creation exists, authenticated multi-user tenant
isolation is mandatory. Native CTOX executes the Web-Stack task graph emitted
by authz run through ctox appsec pipeline enqueue and ctox appsec pipeline work: Secret-Store login, isolated subject crawls, cross-subject replay,
redacted evidence persistence, and matrix import stay in one dependency-bound
worker path. Do not manually recreate that path when native CTOX is available.
Read
references/workflow.md sections 6 and 6.1 before executing those tasks.
Real Source-Aware Audit
Use for a repository, optionally combined with its deployed URL:
pentest --state-dir <state> --tools-root <tools> init --target <repo> [--url <url>] --json
pentest --state-dir <state> --tools-root <tools> audit run --url <url> --source <repo> --json
For granular control, replace the audit run call with
assess --profile full --target <repo> [--url <url>].
Use Greppy for source intelligence and the configured SAST, dependency,
secret, IaC, and configuration scanners for evidence. Do not replace dynamic
validation with source hints. To combine a completed external audit, import
its retained export with artifact import-external; never copy files or mark
coverage complete manually.
Local Lab Acceptance Test
Use only for the bundled intentionally vulnerable localhost lab:
pentest --state-dir <state> init --url http://127.0.0.1:<port> --json
pentest --state-dir <state> scope check --url http://127.0.0.1:<port> --json
pentest --state-dir <state> lab run --url http://127.0.0.1:<port> --profile full --json
lab run is the complete lab orchestrator. Run it once. Do not follow it with
the real external assess flow. Do not run full external-tool readiness or
install missing scanners unless the task separately asks to test installation;
the bundled lab harness supplies its own scanner evidence. The authoritative
lab result still comes from scanner validate, the proof aggregate written by
lab run, artifact audit, and review. Do not rerun finding verify-all
after lab run; that would require a second active-replay confirmation and can
replace valid lab proof state with setup-only records.
After audit run
audit run is the assessment and verification phase, not the completed audit.
Its JSON result carries ok, findings_verified, exploits[],
candidates_unresolved[], coverage_gaps[], and blockers[]. Exit code 0
means the audit ran through (gaps are reported honestly); exit code 1 means a
hard blocker (uninitialized scope, out-of-scope target, missing active
approval, no usable scanner and no seed URLs).
<state>/exploits/index.json (ctox.appsec_pentest.exploit_index.v1) is the
authoritative verified-exploit list: finding id, severity, CWE, target,
script path, script SHA-256, verification status, and provenance per entry
(origin: template-verified or crafted; crafted entries carry a
craft block with the session id/path, iterations, and requests used).
index.md is the rendered table. Never hand-edit either; regenerate by
rerunning.
- Every confirmed exploit was proven by an executed
still-reproduces
verification. Never list scanner candidates as "verified exploits".
candidates_unresolved[] and coverage_gaps[] are open work items, not
decoration. Feed them into the evidence pipeline below; do not close them
by hand or silently drop them from the handoff.
- An unresolved candidate is unresolved work, not a false positive and not a
finding. Manual resolution still goes through the
investigate resolve
gate with its evidence contract.
Then execute the mandatory evidence pipeline against the same explicit state:
Every generated workstream must resolve to a registered scanner or bounded
native probe. manual-validation is rejected legacy capability: regenerate
old pipelines instead of treating a browser seed, narrative, operator review,
or proxy import as completion. Such evidence may supplement an executed
result, but cannot close a workstream by itself.
pentest --state-dir <state> pipeline status --json
pentest --state-dir <state> pipeline work --include-blocked --json
pentest --state-dir <state> artifact audit --json
pentest --state-dir <state> review --json
pentest --state-dir <state> finish \
--executive-summary <text> --methodology <text> \
--technical-analysis <text> --recommendations <text> --json
pentest --state-dir <state> artifact audit --json
pentest --state-dir <state> review --json
pentest --state-dir <state> report --format markdown --out <state>/reports/deployment-audit-report.md --json
Do not claim completion unless the final review JSON has
.completion_review.closable == true and
.completion_review.blocker_count == 0. The full pipeline contract,
dispositions, and completion semantics in the next section apply unchanged.
Evidence Pipeline
After audit run, either granular real orchestrator, or the lab run, execute
these against the same explicit state:
pentest --state-dir <state> pipeline status --json
pentest --state-dir <state> pipeline work --include-blocked --json
pentest --state-dir <state> analyze --json
pentest --state-dir <state> scanner validate --json
pentest --state-dir <state> investigate plan --json
pentest --state-dir <state> investigate list --json
Run the emitted work orders and resolve every scanner candidate. For a
source-aware audit, also disposition every source-review candidate before
closing its stage:
pentest --state-dir <state> pipeline source-decisions --json
pentest --state-dir <state> pipeline source-disposition \
--stage-id <stage-id> --id <candidate-id> \
--status not-exploitable|requires-manual-review|proof-required|validated-vulnerability \
--reason <concrete-reviewed-reason> --artifact <source-or-dataflow-evidence.json> \
[--proof-artifact <executed-proof-verification.json>] --json
pentest --state-dir <state> pipeline source-decisions --json
validated-vulnerability requires --proof-artifact from an executed,
still-reproducing reproduce.py. not-exploitable requires retained source or
dataflow evidence and a concrete reason. requires-manual-review and
proof-required remain unresolved; continue the work rather than closing the
stage. Require .status.closable == true, then bind the decisions artifact to
each affected stage:
pentest --state-dir <state> pipeline rework --stage-id <stage-id> \
--status completed --reason "All source candidates have evidence-backed dispositions" \
--artifact <state>/source-review-decisions.json --json
pentest --state-dir <state> pipeline status --json
Do not proceed while a required stage is blocked or a source candidate is
unresolved. Every stage must be completed or not-applicable with retained
evidence. After all proof execution, dispositions, and rework, regenerate the
authoritative completion state and only then render reports:
pentest --state-dir <state> artifact audit --json
pentest --state-dir <state> review --json
pentest --state-dir <state> finish \
--executive-summary <text> --methodology <text> \
--technical-analysis <text> --recommendations <text> --json
pentest --state-dir <state> artifact audit --json
pentest --state-dir <state> review --json
pentest --state-dir <state> report --format json --out <state>/reports/deployment-audit-report.json --json
pentest --state-dir <state> report --format markdown --out <state>/reports/deployment-audit-report.md --json
Reports consume the current finish.json. Any later disposition, proof,
coverage update, or pipeline rework makes the previous finish stale: rerun
finish, artifact audit, and review before reporting. A closable audit can
correctly block go-live when validated release blockers still reproduce.
For a real audit, execute or re-execute finding proofs only when the rules of
engagement authorize that replay:
CONFIRM_ACTIVE_REPLAY=1 pentest --state-dir <state> finding verify-all --execute --mode http-only --json
For source-only bundles use --mode source-only --source-root <repo>; source
proofs do not require the HTTP active-replay confirmation. finding verify-all
and lab run write reports/proof-verifications.json. Require:
.summary.all_have_executed_proof == true
.summary.invalid == 0
- every validated finding has a verification record
A still-reproducing proof demonstrates a release blocker. After remediation,
the same proof must demonstrate fixed/not-reproducible without changing the
retained before-evidence.
For evidence-bound regression tests, generate the exploit bundle once and keep
it hash-pinned:
pentest --state-dir <state> exploit generate --id <finding-id> --json
pentest --state-dir <state> exploit verify --id <finding-id> --expect vulnerable --execute --confirm-active --json
pentest --state-dir <state> exploit export-test --id <finding-id> --json
exploit generate derives proof_hints.v1 from lab/authz evidence or the
finding's declared proof_hints.v1 and writes
exploit-spec.json with the reproduce.py SHA-256; later verification fails
closed when the proof script drifts. exploit verify --expect asserts the
vulnerable or fixed state with the same execute/confirm gates as
finding verify-bundle. exploit export-test emits a standalone pytest file
for CI (see references/workflow.md section 12 and
references/finding-schema.md for the proof_hints.v1 / exploit_spec.v1
contracts).
For full and deep, every scanner candidate must also have an explicit
investigation outcome. Configure and execute only structured work orders:
pentest --state-dir <state> investigate plan --id <candidate-id> \
--hypothesis <text> --expected-signal <text> \
--falsification-criterion <text> --tool <allowlisted-tool> \
--target <repo> --json
pentest --state-dir <state> investigate execute --id <candidate-id> --json
pentest --state-dir <state> investigate resolve --id <candidate-id> \
--outcome confirmed|refuted|duplicate|not-applicable|inconclusive \
--reason <text> --artifact <candidate-bound-evidence.json> --json
confirmed requires an executed independent work order and candidate-bound
observed_impact evidence, or a valid still-reproduces finding verification
with a hashed HTTP/source proof. Scanner output cannot satisfy this gate.
inconclusive remains visible and blocks full/deep completion.
Before promoting a confirmed candidate, build the deterministic evidence graph
and run the bound refutation gate:
pentest --state-dir <state> graph build --json
pentest --state-dir <state> investigate refute --id <candidate-id> \
--artifact <bound-refutation.json> --json
The graph contains only parsed, hashed facts from scope, scanner triage, proxy
traffic, and imported authz matrices. It stores no LLM-generated edges.
Refutation must bind the current graph and confirmed evidence hashes and check
scope, authentication context, alternative explanations, and reproducer state.
A rejected or inconclusive refutation blocks full/deep completion.
Every assess writes an immutable per-assessment baseline plan before the
first scanner process starts. Retain the returned assessment_id and use the
stored plan for an exact rerun:
pentest --state-dir <state> replay --assessment <assessment-id> --baseline --json
pentest --state-dir <state> replay --assessment <assessment-id> --investigations --json
Baseline replay uses only the recorded scanner work orders. It fails before
starting any process when the scope, arguments, approval, scanner executable,
binary SHA-256, or registry version differs. Investigation replay is
supplemental: it repeats only previously executed, hashed work orders and does
not change the baseline plan or its coverage claim.
Dependency Proof
Dependency candidates from a source-aware assess (osv, trivy, retire)
carry structured installed_version, affected_versions, and
fixed_versions; the remediation text names the concrete fixed release
when known. Prove impact with the four-stage dependency path instead of
reporting scanner versions as findings:
-
Reachability. Annotate every dependency candidate with an
import-level verdict plus file:line call-site evidence:
pentest --state-dir <state> deps reachability [--source <repo>] [--candidate <id>] --json
Verdicts are reachable-import, unreachable, or unknown
(persisted as dependency_reachability.v1). They are advisory: an
unreachable verdict only carries a downgrade-to-informational
recommendation; triage keeps severity ownership.
-
Patch gap. Prove the fix-committed-not-released window when the
installed version predates the fix release:
pentest --state-dir <state> deps patch-gap --repo <path|url> [--installed <version>] [--json]
The scan covers installed-tag..ref for commits with security
signals (CVE/GHSA/keyword catalog) and resolves release status via
git tag --contains. Local repos are read-only; URL clones are
cached and gated behind --allow-network.
-
Patch diff. Classify the root cause of the fix:
pentest --state-dir <state> deps patch-diff --repo <path|url> --from <ref> --to <ref> --json
Persists a hashed dependency_patch_analysis.v1 with the
remediation idiom (output-encoding, sql-injection, command-injection,
path-traversal, deserialization, auth-crypto, ssrf-guard,
input-validation) as the PoC input.
-
PoC synthesis and verification. Render an executable proof from
the patch analysis:
pentest --state-dir <state> deps poc --analysis <patch-diff.json> \
[--module-path <dir>] [--function <name>] [--fixed-version <v>]... \
[--expect vulnerable|fixed] [--execute] --json
deps poc writes a self-contained, hash-pinned dependency_poc.py
(stdlib only): version gate against fixed_versions, dynamic oracle
for the primary root cause. The proof contract matches web proofs
(exit 2/0/1, POC_RESULT JSON), so the same script is red against
the installed version and green after the upgrade. Function hints
come from diff hunk contexts; ambiguous diffs demand --function.
Unsupported root causes yield an honest inconclusive — never a
fabricated finding.
The evidence discipline is unchanged: scanner versions and reachability
verdicts are observations; only the executed PoC promotes a dependency
candidate to a finding.
Completion Semantics
For toolchain-hardening work, inspect the executable requirement and
application-family matrices rather than a raw scanner count:
pentest --state-dir <state> tools requirements --json
pentest --state-dir <state> tools capabilities --json
tools requirements is authoritative for the numbered 1-to-50 hardening
program. Every partial or missing row is an engineering blocker. tools capabilities applies the same rule to each application family: all applicable
required scanners, data bundles, positive/negative fixtures, evidence schemas,
and executable pipelines must be present before the family is complete.
Read the wrapper response shape, not guessed top-level fields:
- Review response:
.completion_review.closable,
.completion_review.blocker_count, .completion_review.blockers
- Investigation state:
.summary.open == 0; each current candidate is
confirmed, refuted, duplicate, or not-applicable
- Full toolchain:
.proof_run.ready from tools proof-run, plus the report's
.toolchain_evidence_summary.full_proof_ready
- Artifact audit:
.ok == true, no findings, no parse errors
- Report: candidates, observations, coverage gaps, product failures, and
validated findings remain visibly separate
For real full/deep audits, full_proof_ready == false blocks a claim of full
scanner readiness. A local lab may close without external scanner readiness
because its bundled harness is the scoped scanner under validation; state this
distinction explicitly.
review is authoritative. If it is not closable, follow its next_commands
or report exact blockers. Never use --allow-incomplete to claim readiness.
In the final handoff, project finding IDs, titles, severities, categories, CWE,
counts, and release decisions directly from the final JSON report. Do not
reconstruct or relabel them from memory, candidates, console summaries, or
earlier output. Verify the per-finding rows reproduce the report's severity
rollup before responding. Distinguish an audit workflow that is closable from
an application that is blocked from go-live by still-reproducing findings.
Active Work And Approvals
Passive mapping, local source analysis, and read-only checks do not authorize
state-changing probes. SQL automation, fuzzing, port scans, cross-account
replays, uploads, race checks, request smuggling, or other active stages require
the applicable approval artifact. Use approval request|grant|status and keep
the decision in audit state. Granted approvals are HMAC-signed with an
operator-held key outside the state dir (~/.config/ctox/appsec-approval-key
or global --approval-key-file <path>); active runs reject unsigned or
tampered approvals, and pre-signing approvals must be re-granted.
For authenticated work, use reference-only ctox-secret:// credential refs.
Login and probe must share the Web-Stack session where the emitted contract
requires it. Inside native CTOX use the pipeline worker; standalone pentest
may expose exact ctox_cli contracts only as an explicit fallback. Import only
redacted authz matrices. Read the matching recipe in references/workflow.md
before creating pipeline inputs.
Exploit Crafting (Template Proof vs. Crafting Session)
A verified exploit starts from the cheapest honest proof and escalates only
when that proof path fails:
- Template proof first. Confirmed findings go through
exploit generate + exploit verify --expect vulnerable --execute; the
exploit index records them with origin: "template-verified".
- Crafting session when the template is refuted or filtered. If the
template payload reflects escaped, a WAF blocks it, or the class needs
adaptation, escalate per class:
craft probe (read-only transformation
oracle) → craft synthesize (candidate hypotheses, no traffic) →
craft run (budgeted iteration engine). A crafted exit records the
finding, generates and verifies the exploit, and registers it in the
exploit index with origin: "crafted" plus a craft block (session id
and path, iterations, requests used).
- Blind injection points (no reflection; signal only via content
difference, latency, or egress) go to
craft extract; choose the oracle
by the matrix in references/workflow.md (auto walks fastest-first:
union → error → boolean → timing → oob).
- Multi-step impact over already validated findings goes to
chain build / chain verify --execute; a chain counts only when its
proof script exits 2 against the live target.
Staged approvals gate the escalation: craft probe and craft synthesize
stay approval-free (canary-grade); craft run with sqli/ssti/all requires a
granted approval with --craft-stage run or higher; craft extract always
requires stage extract; chain verify requires stage chain. Request the
stage with approval request --craft-stage <stage>; a higher-stage approval
covers lower stages, and pre-hierarchy approvals without a craft_stage
field count as legacy run. Extraction stays data-minimized
(--max-chars/--length only what the proof needs; --reveal-into is
opt-in).
Proof discipline is absolute: never report an unproven synthesis as a
finding. Payload candidates from craft synthesize, a winning iteration
without a successful exploit verify, and scanner output are hypotheses.
exhausted, no-candidates, no-oracle, and unstable-oracle are honest
outcomes — record them as coverage notes, never as findings.
Read crafting sessions at <state>/crafting/session-*/session.json
(crafting_session.v1): target, class, budget, approval_ref,
profile_ref/candidates_ref (path + sha256), iterations[]
(candidate_id, payload, outcome = success/refuted/filtered/baseline,
signal, request_sha256 — bodies are hashed, never inlined),
requests_used, and result. The full recipes, oracle matrix, and
exit-status contracts live in references/workflow.md (§13).
Reference Loading
Read only what the current phase needs:
references/workflow.md: end-to-end recipes, Web-Stack authz, pipeline
evidence, finding validation, remediation, and reporting.
references/tool-policy.md: tool allowlist, installation, raw arguments,
scanner safety, state handling, and missing-tool policy.
references/finding-schema.md: finding and proof-bundle fields.
references/strix-port-map.md: Strix-to-CLI capability mapping.
references/strix/: vulnerability, framework, technology, protocol,
scanner, and scan-mode playbooks. Load only the relevant file.
The CLI remains the scope, state, approval, evidence, and completion boundary
regardless of which reference guides the validation technique.