clawsec-scanner

# ClawSec Scanner Comprehensive security scanner for agent platforms that automates vulnerability detection across multiple dimensions: - **Dependency Scanning**: Analyzes npm and Python dependencies using `npm audit` and `pip-audit` with structured JSON output parsing - **CVE Database Integration**: Queries OSV (primary), NVD 2.0, and GitHub Advisory Database for vulnerability enrichment - **SAST Analysis**: Static code analysis using Semgrep (JavaScript/TypeScript) and Bandit (Python) to detect hardcoded secrets, command injection, path traversal, and unsafe deserialization - **DAST Framework**: Agent-specific static analysis of OpenClaw hook metadata and handler source without importing or invoking target code - **Unified Reporting**: Consolidated vulnerability reports with severity classification and remediation guidance - **Continuous Monitoring**: OpenClaw hook integration for automated periodic scanning ## Vercel Skills Installation Install with the Vercel Skills CLI for this harness: ```bash npx skills add prompt-security/clawsec --skill clawsec-scanner -a openclaw -y ``` ## Features ### Multi-Engine Scanning The scanner orchestrates four complementary scan types to provide comprehensive vulnerability coverage: 1. **Dependency Scanning** - Executes `npm audit --json` and `pip-audit -f json` as subprocesses - Parses structured output to extract CVE IDs, severity, affected versions - Handles edge cases: missing package-lock.json, zero vulnerabilities, malformed JSON 2. **CVE Database Queries** - **OSV API** (primary): Free, no authentication, broad ecosystem support (npm, PyPI, Go, Maven) - **NVD 2.0** (optional): Requires API key to avoid 6-second rate limiting - **GitHub Advisory Database** (optional): GraphQL API with OAuth token - Normalizes all API responses to unified `Vulnerability` schema 3. **Static Analysis (SAST)** - **Semgrep** for JavaScript/TypeScript: Detects security issues using `--config auto` or `--config p/security-audit` - **Bandit** for Python: Leverages existing `pyproject.toml` configuration - Identifies: hardcoded secrets (API keys, tokens), command injection (`eval`, `exec`), path traversal, unsafe deserialization 4. **Dynamic Analysis (DAST)** - Static hook inspection for OpenClaw hook handlers discovered from `HOOK.md` metadata - Verifies coverage and source-level risk signals without importing, transpiling, or invoking target handlers - Note: Traditional web DAST tools (ZAP, Burp) do not apply to agent platforms - this provides agent-specific testing ### Unified Reporting All scan types emit a consistent `ScanReport` JSON schema: ```typescript { scan_id: string; // UUID timestamp: string; // ISO 8601 target: string; // Scanned path vulnerabilities: Vulnerability[]; summary: { critical: number; high: number; medium: number; low: number; info: number; } } ``` Each `Vulnerability` object includes: - `id`: CVE-2023-12345 or GHSA-xxxx-yyyy-zzzz - `source`: npm-audit | pip-audit | osv | nvd | github | sast | dast - `severity`: critical | high | medium | low | info - `package`: Package name (or 'N/A' for SAST/DAST) - `version`: Affected version - `fixed_version`: First version with fix (if available) - `title`: Short description - `description`: Full advisory text - `references`: URLs for more info - `discovered_at`: ISO 8601 timestamp ### OpenClaw Integration Automated continuous monitoring via hook: - Runs scanner on configurable interval (default: 86400s / 24 hours) - Triggers on `agent:bootstrap` and `command:new` events - Posts findings to `event.messages` array with severity summary - Rate-limited by `CLAWSEC_SCANNER_INTERVAL` environment variable ## Installation ### Prerequisites Verify required binaries are available: ```bash # Core runtimes node --version # v20+ npm --version python3 --version # 3.10+ # Scanning tools pip-audit --version # Install: uv pip install pip-audit semgrep --version # Install: pip install semgrep OR brew install semgrep bandit --version # Install: uv pip install bandit # Utilities jq --version curl --version ``` ### Option A: Via clawhub (recommended) ```bash npx clawhub@latest install clawsec-scanner ``` ### Option B: Manual installation with verification ```bash set -euo pipefail VERSION="${SKILL_VERSION:?Set SKILL_VERSION (e.g. 0.1.0)}" INSTALL_ROOT="${INSTALL_ROOT:-$HOME/.openclaw/skills}" DEST="$INSTALL_ROOT/clawsec-scanner" BASE="https://github.com/prompt-security/clawsec/releases/download/clawsec-scanner-v${VERSION}" TEMP_DIR="$(mktemp -d)" trap 'rm -rf "$TEMP_DIR"' EXIT # Pinned release-signing public key # Fingerprint (SHA-256 of SPKI DER): 711424e4535f84093fefb024cd1ca4ec87439e53907b305b79a631d5befba9c8 cat > "$TEMP_DIR/release-signing-public.pem" <<'PEM' -----BEGIN PUBLIC KEY----- MCowBQYDK2VwAyEAS7nijfMcUoOBCj4yOXJX+GYGv2pFl2Yaha1P4v5Cm6A= -----END PUBLIC KEY----- PEM ZIP_NAME="clawsec-scanner-v${VERSION}.zip" # Download release archive + signed checksums curl -fsSL "$BASE/$ZIP_NAME" -o "$TEMP_DIR/$ZIP_NAME" curl -fsSL "$BASE/checksums.json" -o "$TEMP_DIR/checksums.json" curl -fsSL "$BASE/checksums.sig" -o "$TEMP_DIR/checksums.sig" # Verify checksums manifest signature openssl base64 -d -A -in "$TEMP_DIR/checksums.sig" -out "$TEMP_DIR/checksums.sig.bin" if ! openssl pkeyutl -verify \ -pubin \ -inkey "$TEMP_DIR/release-signing-public.pem" \ -sigfile "$TEMP_DIR/checksums.sig.bin" \ -rawin \ -in "$TEMP_DIR/checksums.json" >/dev/null 2>&1; then echo "ERROR: checksums.json signature verification failed" >&2 exit 1 fi EXPECTED_SHA="$(jq -r '.archive.sha256 // empty' "$TEMP_DIR/checksums.json")" if [ -z "$EXPECTED_SHA" ]; then echo "ERROR: checksums.json missing archive.sha256" >&2 exit 1 fi ACTUAL_SHA="$(shasum -a 256 "$TEMP_DIR/$ZIP_NAME" | awk '{print $1}')" if [ "$EXPECTED_SHA" != "$ACTUAL_SHA" ]; then echo "ERROR: Archive checksum mismatch" >&2 exit 1 fi echo "Checksums verified. Installing..." mkdir -p "$INSTALL_ROOT" rm -rf "$DEST" unzip -q "$TEMP_DIR/$ZIP_NAME" -d "$INSTALL_ROOT" chmod 600 "$DEST/skill.json" find "$DEST" -type f ! -name "skill.json" -exec chmod 644 {} \; echo "Installed clawsec-scanner v${VERSION} to: $DEST" echo "Next step: Run a scan or set up continuous monitoring" ``` ## Usage ### On-Demand CLI Scanning ```bash SCANNER_DIR="${INSTALL_ROOT:-$HOME/.openclaw/skills}/clawsec-scanner" # Scan all skills with JSON output "$SCANNER_DIR/scripts/runner.sh" --target ./skills/ --output report.json --format json # Scan specific directory with human-readable output "$SCANNER_DIR/scripts/runner.sh" --target ./my-skill/ --format text # Check available flags "$SCANNER_DIR/scripts/runner.sh" --help ``` **CLI Flags:** - `--target <path>`: Directory to scan (required) - `--output <file>`: Write results to file (optional, defaults to stdout) - `--format <json|text>`: Output format (default: json) - `--check`: Verify all required binaries are installed ### OpenClaw Hook Setup (Continuous Monitoring) Enable automated periodic scanning: ```bash SCANNER_DIR="${INSTALL_ROOT:-$HOME/.openclaw/skills}/clawsec-scanner" node "$SCANNER_DIR/scripts/setup_scanner_hook.mjs" ``` This creates a hook that: - Scans on `agent:bootstrap` and `command:new` events - Respects `CLAWSEC_SCANNER_INTERVAL` rate limiting (default: 86400 seconds / 24 hours) - Posts findings to conversation with severity summary - Recommends remediation for high/critical vulnerabilities Restart the OpenClaw gateway after enabling the hook, then run `/new` to trigger an immediate scan. ### Environment Variables ```bash # Optional - NVD API key to avoid rate limiting (6-second delays without key) export CLAWSEC_NVD_API_KEY="your-nvd-api-key" # Optional - GitHub OAuth token for Advisory Database queries export GITHUB_TOKEN="ghp_your_token_here" # Optional - Scanner hook interval in seconds (default: 86400 / 24 hours) export CLAWSEC_SCANNER_INTERVAL="86400" # Optional - Allow unsigned advisory feed during development (from clawsec-suite) export CLAWSEC_ALLOW_UNSIGNED_FEED="1" ``` ## Architecture ### Modular Design Each scan type is an independent module that can run standalone or as part of unified scan: ``` scripts/runner.sh # Orchestration layer ├── scan_dependencies.mjs # npm audit + pip-audit ├── query_cve_databases.mjs # OSV/NVD/GitHub API queries ├── sast_analyzer.mjs # Semgrep + Bandit static analysis ├── dast_runner.mjs # Static hook inspection orchestration └── dast_hook_executor.mjs # Static hook source inspection helper lib/ ├── report.mjs # Result aggregation and formatting ├── utils.mjs # Subprocess exec, JSON parsing, error handling └── types.ts # TypeScript schema definitions hooks/clawsec-scanner-hook/ ├── HOOK.md # OpenClaw hook metadata └── handler.ts # Periodic scan trigger ``` ### Fail-Open Philosophy The scanner prioritizes availability over strict failure propagation: - Network failures → emit partial results, log warnings - Missing tools → skip that scan type, continue with others - Malformed JSON → parse what's valid, log errors - API rate limits → implement exponential backoff, fallback to other sources - Zero vulnerabilities → emit success report with empty array **Critical failures** that exit immediately: - Target path does not exist - No scanning tools available (all bins missing) - Concurrent scan detected (lockfile present) ### Subprocess Execution Pattern All external tools run as subprocesses with structured JSON output: ```javascript import { spawn } from 'node:child_process'; // Example: npm audit execution const proc = spawn('npm', ['audit', '--json'], { cwd: targetPath, stdio: ['ignore', 'pipe', 'pipe'] }); // Handle non-zero exit codes gracefully // npm audit exits 1 when vulnerabilities found (not an error!) proc.on('close', code => { if (code !== 0 && stderr.includes('ERR!')) { // Actual error reject(new Error(stderr)); } else { // Vulnerabilities found or success resolve(JSON.parse(stdout)); } }); ``` ## Troubleshooting ### Common Issues **"Missing package-lock.json" warning** - `npm audit` requires lockfile to run - Run `npm install` in target directory to generate - Scanner continues with other scan types if npm audit fails **"NVD API rate limit exceeded"** - Set `CLAWSEC_NVD_API_KEY` environment variable - Without API key: 6-second delays enforced between requests - OSV API used as primary source (no rate limits) **"pip-audit not found"** - Install: `uv pip install pip-audit` or `pip install pip-audit` - Verify: `which pip-audit` - Add to PATH if installed in non-standard location **"Semgrep binary missing"** - Install: `pip install semgrep` OR `brew install semgrep` - Requires Python 3.8+ runtime - Alternative: use Docker image `returntocorp/semgrep` **"DAST static coverage finding"** - The DAST harness does not execute target hook handlers. - JavaScript and TypeScript hook files are read as source and reported with `info`-level static coverage findings. - Review any listed static signals manually when deciding whether a hook needs deeper sandboxed testing. **"Concurrent scan detected"** - Lockfile exists: `/tmp/clawsec-scanner.lock` - Wait for running scan to complete or manually remove lockfile - Prevents overlapping scans that could produce inconsistent results ### Verification Check scanner is working correctly: ```bash # Verify required binaries ./scripts/runner.sh --check # Run unit tests node test/dependency_scanner.test.mjs node test/cve_integration.test.mjs node test/sast_engine.test.mjs node test/dast_harness.test.mjs # Validate skill structure python ../../utils/validate_skill.py . # Scan test fixtures (should detect known vulnerabilities) ./scripts/runner.sh --target test/fixtures/ --format text ``` ## Development ### Running Tests ```bash # All tests (vanilla Node.js, no framework) for test in test/*.test.mjs; do node "$test" || exit 1 done # Individual test suites node test/dependency_scanner.test.mjs # Dependency scanning node test/cve_integration.test.mjs # CVE database APIs node test/sast_engine.test.mjs # Static analysis node test/dast_harness.test.mjs # DAST static hook inspection ``` ### Linting ```bash # JavaScript/TypeScript npx eslint . --ext .ts,.tsx,.js,.jsx,.mjs --max-warnings 0 # Python (Bandit already configured in pyproject.toml) ruff check . bandit -r . -ll # Shell scripts shellcheck scripts/*.sh ``` ### Adding Custom Semgrep Rules Create custom rules in `.semgrep/rules/`: ```yaml rules: - id: custom-security-rule pattern: dangerous_function($ARG) message: Avoid dangerous_function - use safe_alternative instead severity: WARNING languages: [javascript, typescript] ``` Update `scripts/sast_analyzer.mjs` to include custom rules: ```javascript const proc = spawn('semgrep', [ 'scan', '--config', 'auto', '--config', '.semgrep/rules/', // Add custom rules '--json', targetPath ]); ``` ## Integration with ClawSec Suite The scanner works standalone or as part of the ClawSec ecosystem: - **clawsec-suite**: Meta-skill that can install and manage clawsec-scanner - **clawsec-feed**: Advisory feed for malicious skill detection (complementary) - **openclaw-audit-watchdog**: Cron-based audit automation (similar pattern) Install the full ClawSec suite: ```bash npx clawhub@latest install clawsec-suite