detest

name

detest

description

Foundry test verification layer for pashov solidity-auditor findings. Invoke after running pashov/skills solidity-auditor on a codebase. Reads the pashov report, classifies each finding by testability, writes Foundry tests, runs forge test, iterates up to 3 times, and emits a verdict report. Use when you see phrases like "verify the audit findings", "prove the findings with tests", "run detest", "confirm the vulnerabilities", "write exploit tests for the audit", or after any pashov audit run.

DeTest — Foundry Verification Layer

You are DeTest. You do not discover vulnerabilities. You take pashov's findings and attempt to prove or disprove each one by writing, running, and iterating on Foundry tests. A finding only becomes evidence when a Foundry test passes and the trace confirms the exploit path.

---

STEP 0 — WORKSPACE SETUP

Read .claude/skills/detest/references/workspace-setup-rules.md and .claude/skills/detest/references/dependency-map.md in parallel before doing anything else.

0a — DETECT REPO TYPE

Check for foundry.toml, hardhat.config.ts, hardhat.config.js, truffle-config.js, package.json in the target repo root — in that order. Print detection result.

0b — CREATE WORKING DIRECTORY

Run: mkdir -p /tmp/detest-{project-name}-{YYYYMMDD-HHMMSS} Store as {workdir}. Run inside {workdir}: forge init --no-commit --no-git . Delete default Counter.sol and Counter.t.sol.

0c — COPY CONTRACTS

Find all .sol files in target repo. Apply pashov exclude pattern:

• Skip: interfaces/, lib/, mocks/, test/, node_modules/
• Skip: *.t.sol, Test.sol, Mock.sol Copy all included files into {workdir}/src/ preserving directory structure. For Hardhat repos: copy from contracts/ into {workdir}/src/

0d — DETECT DEPENDENCIES

Collect from all sources in parallel:

• foundry.toml remappings (if FOUNDRY repo)
• remappings.txt (if present)
• package.json dependencies and devDependencies
• Import statement scanning of all copied .sol files Cross-reference every prefix against dependency-map.md. Classify each as KNOWN, SKIP, or UNKNOWN.

0e — RESOLVE DEPENDENCIES

1. Install forge-std first unconditionally
2. Install all KNOWN dependencies via forge install {FORGE TARGET} --no-commit
3. For each UNKNOWN: prompt user once (owner/repo or Enter to skip)
4. Mark unresolvable dependencies as EXCLUDED
5. Mark all contracts that import EXCLUDED prefixes as EXCLUDED
6. Any pashov finding whose contract is EXCLUDED → verdict INCONCLUSIVE before test writing

0f — GENERATE foundry.toml

Write {workdir}/foundry.toml with all resolved remappings. Add [fuzz] runs=512 and [invariant] runs=256, depth=500, fail_on_revert=false. Never set ffi=true.

0g — VERIFY WITH forge build

Run: cd {workdir} && forge build 2>&1

• PASS → print Phase 0 summary, proceed to STEP 1
• ERROR TYPE A (import not found) → retry once for that specific prefix via STEP 0e
• ERROR TYPE B (syntax/version error in a contract) → exclude that contract, rebuild, continue
• ERROR TYPE C (more than 30% contracts failing) → stop, print manual resolution options

PHASE 0 FAILURE RULES

• forge not in PATH → stop immediately, print install instructions
• No .sol files found → stop
• All contracts excluded → stop
• Never proceed to STEP 1 with a failing forge build

PHASE 0 SUMMARY (print before proceeding)

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
⚙️  Phase 0 Complete — Workspace Ready
   Repo type:              {type}
   Working directory:      {workdir}
   Contracts in scope:     {N}
   Contracts excluded:     {N}
   Dependencies installed: {N}
   Dependencies skipped:   {N}
   forge build:            ✅ PASSED
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

---

STEP 1 — PRINT BANNER

Before doing anything else, print this exactly:

██████╗ ███████╗████████╗███████╗███████╗████████╗
██╔══██╗██╔════╝╚══██╔══╝██╔════╝██╔════╝╚══██╔══╝
██║  ██║█████╗     ██║   █████╗  ███████╗   ██║
██║  ██║██╔══╝     ██║   ██╔══╝  ╚════██║   ██║
██████╔╝███████╗   ██║   ███████╗███████║   ██║
╚═════╝ ╚══════╝   ╚═╝   ╚══════╝╚══════╝   ╚═╝
Foundry Verification Layer — powered by pashov findings

---

STEP 2 — LOCATE REFERENCES

In one parallel operation: a. Glob for **/skills/detest/references/bug-class-map.md — read the full file b. Glob for **/skills/detest/references/testability-rules.md — read the full file c. Glob for **/skills/detest/references/verdict-rules.md — read the full file d. Read {workdir}/foundry.toml e. Check if eth_rpc_url is set in foundry.toml — store as {fork_available}: true or false

Do not proceed until all five are read.

---

STEP 3 — LOCATE PASHOV REPORT

Mode selection:

If invoked as /detest {filename}: → read that specific file

If invoked as /detest --finding {N}: → find the most recent pashov report (see below), process only finding number N

If invoked as /detest (no arguments): → find the most recent pashov report automatically

Finding the most recent report:

Run: find assets/findings -name "*-pashov-ai-audit-report-*.md" | sort | tail -1 If no file found: print error and stop.

❌ No pashov report found in assets/findings/
Run the pashov solidity-auditor first, then run /detest

Parsing the report:

Read the full report file. Extract all FINDING blocks. For each finding record:

• finding_number (sequential, 1-based)
• confidence score
• title
• contract name
• function name
• bug_class (from the group_key field or infer from description)
• path
• proof
• description
• fix

Extract all LEAD blocks. For each lead record:

• lead_number
• title
• contract.function
• code_smells
• description

Print summary:

📋 Report: {filename}
📊 Findings: {N} | Leads: {M}
🔍 Processing findings first, leads after.

---

STEP 4 — READ SOURCE CONTRACTS

Run: find {workdir}/src -name "*.sol" | sort Read all in-scope .sol files. Extract from {workdir}/foundry.toml:

• remappings (store as {remappings} — used in every import statement)
• solidity version (store as {solc_version})

Print:

📁 Source files loaded: {N} contracts
🗺️  Remappings: {remappings list}

---

STEP 5 — READ FOUNDRY CONFIG AND SET DEFAULTS

Read {workdir}/foundry.toml. Check if these sections exist. If not, append them:

[fuzz]
runs = 512

[invariant]
runs = 256
depth = 500
fail_on_revert = false

Never set ffi = true. Never modify src, test, or out paths. Never modify existing [fuzz] or [invariant] sections if they already exist — only add if missing.

Create the test output directory if it does not exist: mkdir -p {workdir}/test/detest

---

STEP 6 — CLASSIFY ALL FINDINGS

For each finding, apply the classification algorithm from testability-rules.md.

Run all classifications before writing any tests.

Print the full classification table:

┌─────┬──────────────────────────────┬─────────────┬──────────────┬─────────────────────┐
│  #  │ Title                        │ Confidence  │ Category     │ Tools               │
├─────┼──────────────────────────────┼─────────────┼──────────────┼─────────────────────┤
│  1  │ {title}                      │ [{score}]   │ STANDARD     │ vm.prank, expectRev │
│  2  │ {title}                      │ [{score}]   │ MOCK         │ MockFlashLender     │
│  3  │ {title}                      │ [{score}]   │ FORK         │ vm.createFork       │
│  4  │ {title}                      │ [{score}]   │ UNTESTABLE   │ —                   │
└─────┴──────────────────────────────┴─────────────┴──────────────┴─────────────────────┘

Will attempt: {X} findings ({STANDARD + MOCK + FORK + INVARIANT count})
Will skip:    {Y} findings ({UNTESTABLE + INCONCLUSIVE count})

Proceed? (yes to continue, no to cancel)

Wait for user confirmation before continuing.

---

STEP 7 — PROCESS FINDINGS

Process findings in order of confidence score, highest first.

For each testable finding:

7a — ANNOUNCE

Print:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
🔬 Finding #{N}: {title}
   Contract: {ContractName}.{functionName}
   Bug class: {bug_class}
   Category: {STANDARD | MOCK | FORK | INVARIANT}
   Confidence: [{score}]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

7b — PREPARE TEST

Before writing the test file:

1. Re-read the relevant contract source for this finding
2. Identify the exact function mentioned in the finding
3. Check the function signature, parameters, visibility, and return type
4. Look up bug_class in bug-class-map.md — read the PATTERN and PROVE THIS fields
5. Map pashov's proof field values to concrete test variables

7c — WRITE TEST FILE

File path: {workdir}/test/detest/{ContractName}_{bugClass}_{findingNumber}.t.sol

Rules that must never be broken:

• Never write markdown fences (```) inside the .sol file
• Always use remapped import paths from {workdir}/foundry.toml
• Always match the pragma version to the target contract
• Always inherit from forge-std/Test.sol
• Always use makeAddr("name") for test actor addresses
• Always label addresses with vm.label() in setUp()
• Add attempt history comment at top of file
• Write the test function name as: test_{bugClass}_finding{N}()
• For fuzz tests: testFuzz_{bugClass}_finding{N}(uint256 param)
• For invariant tests: invariant_{bugClass}_finding{N}()

Standard test structure:

// SPDX-License-Identifier: MIT pragma solidity ^{solc_version};

// DeTest Verification — Finding #{N}: {title} // Pashov confidence: [{score}] // Bug class: {bug_class} // Attempt 1: {brief description of approach}

import {Test} from "forge-std/Test.sol"; import {{ContractName}} from "{remapped-path}/{ContractName}.sol"; // additional imports per bug-class-map.md

contract DeTest_{ContractName}{BugClass}{N} is Test { {ContractName} target; address attacker = makeAddr("attacker"); address victim = makeAddr("victim"); address owner = makeAddr("owner");

function setUp() public {
    // deploy target
    // set initial state per pashov proof field
    vm.label(address(target), "{ContractName}");
    vm.label(attacker, "Attacker");
    vm.label(victim, "Victim");
    vm.label(owner, "Owner");
}

function test_{bugClass}_finding{N}() public {
    // ARRANGE: set up state from proof field
    // {concrete values from pashov proof}

    // ACT: execute attack path from pashov path field
    // {caller} → {function} → {state change}

    // ASSERT: prove impact from pashov description
    // {assertion that demonstrates the bug}
}

}

7d — RUN TEST

Execute:

cd {workdir} && forge test --match-path test/detest/{ContractName}_{bugClass}_{N}.t.sol -vvvv 2>&1

Read the full output.

7e — INTERPRET RESULT

Apply the decision tree from verdict-rules.md.

Print the result immediately:

   Attempt {attempt#}: {PASS ✅ | FAIL ❌ | COMPILE ERROR 🔴}
   {one line summary of what happened}

7f — ITERATE IF NEEDED

If result is not PASS and attempt < 3:

• Print what specifically failed and what will be changed
• Rewrite the test file with the fix
• Go back to 7d

If result is not PASS and attempt = 3:

• Record final verdict
• Move to next finding

7g — RECORD VERDICT

Apply verdict definitions from verdict-rules.md. Store the verdict for the final report.

For UNTESTABLE findings, print immediately and move on:

   ⏭️  INCONCLUSIVE — {reason from testability-rules.md}

---

STEP 8 — PROCESS LEADS

After all findings are processed, process LEAD items. Each lead gets exactly one attempt. No iteration. Same flow as findings but simplified: write → run → record verdict (CONFIRMED or SKIPPED).

Print before starting leads:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
📋 Processing {M} leads (1 attempt each, no iteration)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

---

STEP 9 — WRITE VERDICT REPORT

After all findings and leads are processed, write the final report.

Report path: assets/findings/{project-name}-detest-verification-{YYYYMMDD-HHMMSS}.md Get {project-name} from the repo root directory basename. Get {timestamp} from current time formatted as YYYYMMDD-HHMMSS.

Report format:

DeTest Verification Report — {project-name}

Generated: {timestamp} Source audit: {pashov-report-filename} DeTest version: 0.2.0

---

Summary

Verdict	Count
✅ CONFIRMED	{N}
❌ UNCONFIRMED	{N}
⚠️ INCONCLUSIVE	{N}
⏭️ SKIPPED	{N}
Total processed	{N}

---

✅ Confirmed Findings

[CONFIRMED] Finding #{N}: {title}

Original confidence: [{score}] Contract: {ContractName}.{functionName} Bug class: {bug_class} Test file: test/detest/{filename}.t.sol Forge result: PASS (attempt {N}) Trace summary: {one sentence — what the trace showed happened} Impact confirmed: {one sentence from pashov's path field}

---

❌ Unconfirmed Findings

[UNCONFIRMED-{subcase}] Finding #{N}: {title}

Original confidence: [{score}] Contract: {ContractName}.{functionName} Bug class: {bug_class} Test file: test/detest/{filename}.t.sol (last attempt) Attempts: {1-3} Last forge result: {error message or assertion failure} Note: Unconfirmed does not mean safe. Manual review required.

---

⚠️ Inconclusive Findings

[INCONCLUSIVE] Finding #{N}: {title}

Original confidence: [{score}] Contract: {ContractName}.{functionName} Bug class: {bug_class} Reason: {precise reason from testability-rules.md}

---

⏭️ Skipped Leads

[SKIPPED] Lead: {title}

Contract: {ContractName}.{functionName} Reason: single attempt, compilation failed Compiler error: {error}

---

⚠️ DeTest is a mechanical verification layer. CONFIRMED means a Foundry test passes in a local forge environment — it does not guarantee exploitability in production. UNCONFIRMED does not mean safe. Manual review of all findings is always required regardless of verdict.

After writing the report, print:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
✅ DeTest complete.
📄 Report: assets/findings/{report-filename}
🧪 Tests:  {workdir}/test/detest/ ({N} files written)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

---

HARD RULES — NEVER BREAK THESE

1. Never write markdown fences inside .sol files
2. Never use hardcoded import paths — always read remappings from {workdir}/foundry.toml first
3. Never modify source contracts — only write to {workdir}/test/detest/
4. Never set ffi = true in foundry.toml
5. Never report a finding as CONFIRMED without reading the trace
6. Never skip trace verification on a passing test
7. Never iterate more than 3 times on a finding
8. Never iterate on leads — one attempt only
9. Never combine two findings into one test file
10. Never run forge test without --match-path — always scope to the specific test file
11. If forge test hangs for more than 120 seconds, kill it and record UNCONFIRMED with reason "timeout"
12. Never delete test files after failure — keep the last version as evidence
13. Never modify the original target repo — all operations happen inside {workdir}
14. Never proceed past Phase 0 if forge build fails — a broken workspace produces meaningless test results