# ClawScan 📡
ClawScan is a composable security scanning harness for agent skills.
Run a suite of skill security scanners, pass the results to a judge harness, and compare against multiple skill security benchmarks.
[](https://github.com/openclaw/clawscan/actions/workflows/ci.yml?query=branch%3Amain)
[](https://github.com/openclaw/clawscan/actions/workflows/release.yml)
[](https://github.com/openclaw/clawscan/releases)
## Quick Start
Install ClawScan:
```bash
npm install -g @openclaw/clawscan
```
The macOS binaries support macOS 13 or later on Apple Silicon and Intel. CI uses
the Apple Silicon `xcode-27` runner and verifies x86_64 binaries under Rosetta.
Release and npm builds pin the macOS 13 deployment target and check both binaries' minimum OS metadata.
Source builds require Go 1.27.1 or later; the module selects the patched Go 1.27.3 build toolchain.
Command-backed scanners and judges run in ClawScan's Docker runtime by default,
so keep Docker running for local scans.
Run NVIDIA SkillSpector and Cisco Skill Scanner against a local `skills/` folder:
```bash
clawscan --scanner skillspector --scanner cisco
```
## Scan a known malicious skill
This example scans Trail of Bits' [`csv-summarizer`](https://github.com/trailofbits/overtly-malicious-skills/tree/4ffbf9461ef0505f9ce76a0d3694a18ec33ea531/skills/csv-summarizer) skill, which claims to summarize a CSV file but also prints every environment variable when run.
```bash
git clone https://github.com/trailofbits/overtly-malicious-skills.git /tmp/overtly-malicious-skills
cd /tmp/overtly-malicious-skills
git checkout 4ffbf9461ef0505f9ce76a0d3694a18ec33ea531
clawscan skills/csv-summarizer \
--scanner skillspector \
--scanner cisco \
--output /tmp/clawscan-csv-summarizer.json
```
Sample findings:
```txt
targets: 1
scanner_completed: 2
scanner_failed: 0
scanner_skipped: 0
issues_found: 2
gate: pass
errors: 0
full_results: /tmp/clawscan-csv-summarizer.json
```
The results bundle keeps the top-level artifact plus per-scanner JSON reports.
<details>
<summary>Artifact excerpt</summary>
```json
{
"schemaVersion": "clawscan-run-v1",
"target": "skills/csv-summarizer",
"scanners": {
"cisco": {
"status": "completed",
"durationMs": 42,
"outputPath": "clawscan-csv-summarizer/skills/csv-summarizer/cisco.json",
"isSafe": true,
"maxSeverity": "SAFE",
"findingsCount": 0
},
"skillspector": {
"status": "completed",
"durationMs": 42,
"outputPath": "clawscan-csv-summarizer/skills/csv-summarizer/skillspector.json",
"severity": "MEDIUM",
"score": 31,
"recommendation": "CAUTION",
"issues": [
{
"id": "LP3",
"severity": "MEDIUM",
"file": "SKILL.md"
},
{
"id": "E2",
"severity": "HIGH",
"file": "scripts/summarize.py"
}
]
}
}
}
```
</details>
## Motivation
Agent-skill security is new and fast-moving, with researchers and companies
exploring many promising scanners, datasets, and judge harnesses. In our
[ClawHub Security Signals paper](https://arxiv.org/html/2606.01494v1), we found
that combining multiple scanners with a configurable judge works better than
relying on any single scanner.
ClawScan turns that approach into a repeatable CLI. It includes a built-in `clawhub` profile, a saved scanner-and-judge configuration that matches what ClawHub runs in production, so researchers can reproduce results, test improvements, and help improve detection against the weekly refreshed ClawHub security-signals dataset.
## Commands
| Command family | Use |
| --- | --- |
| `clawscan <target> --scanner <id>` | Run one or more scanners against an explicit skill or OpenClaw plugin target. Omit `<target>` to scan child skill directories under `./skills`; plugins are never auto-discovered. |
| `clawscan scanners [list\|<scanner-id>]` | Discover supported scanner IDs, required env vars, upstream links, descriptions, and install guidance. |
| `clawscan profiles [-v]` | Inspect built-in profiles; `-v` prints the catalog as YAML. |
| `clawscan benchmark [list\|<benchmark-id>]` | Discover or run supported benchmarks through a selected scanner/profile/judge setup. |
| `clawscan install <scanner-id> [...]` | Install or verify local scanner dependencies where ClawScan has registry-backed install plans. |
| `clawscan openclaw-install-policy` | Act as an external OpenClaw `security.installPolicy.exec` command. Reads the staged install request from stdin and returns allow/warn/block JSON. |
## OpenClaw install policy
ClawScan integrates with OpenClaw at the operator-owned
`security.installPolicy` boundary. It does not register an install hook or
depend on plugin activation. The policy command scans the staged `sourcePath`
for both skills and plugins before OpenClaw commits a supported install or
update.
See [docs/openclaw-install-policy.md](docs/openclaw-install-policy.md) for the
trusted executable setup, configuration, payload contract, and scope.
## Scanners
`--scanner` selects a scanner adapter to run, writes its raw JSON evidence into
the results artifact, and can be repeated to compare multiple scanners in one
run:
```bash
clawscan ./my-skill \
--scanner skillspector \
--scanner cisco
```
Discover the scanner catalog from the CLI:
```bash
clawscan scanners
clawscan scanners skillspector
```
### Available scanners
> **Want to add your scanner to the list?** Follow the guide in [docs/scanners.md](docs/scanners.md#adding-a-built-in-scanner-adapter)
| ID | Name | Repo | Description | Required env vars | Local dependency setup |
| --- | --- | --- | --- | --- | --- |
| `agentverus` | AgentVerus | [repo](https://github.com/agentverus/agentverus-scanner) | Local file or directory scanner invoked through agentverus-scanner. | none | `npm install --save-dev agentverus-scanner` |
| `aig` | Tencent AI-Infra-Guard | [repo](https://github.com/Tencent/AI-Infra-Guard/tree/main/skill-scan) | Tencent Zhuque Lab's local directory scanner invoked through `aig-skill-scan`. Produces SARIF 2.1.0 with SkillTrustBench T01-T09 evidence. | `LLM_API_KEY` or `OPENAI_API_KEY`<br><details><summary>Optional config</summary><code>DEFAULT_MODEL</code>, <code>REASONING_EFFORT</code> (requires the patched runtime), <code>DEFAULT_BASE_URL</code>, <code>DEFAULT_MODEL_CONTEXT_WINDOW</code>, <code>LOG_LEVEL</code>.</details> | `pip install aig-skill-scan` |
| `aig-mcp` | Tencent AI-Infra-Guard MCP scanner | [repo](https://github.com/Tencent/AI-Infra-Guard/tree/main/mcp-scan) | Tencent Zhuque Lab's LLM-driven MCP server code audit invoked through `aig-mcp-scan`, run on OpenClaw plugin targets only (skill, file and URL targets are skipped). Audits a bounded temporary copy of the whole plugin without `node_modules`, using an OpenClaw plugin audit prompt, and produces SARIF 2.1.0 with MCP01-MCP10 evidence. Runs in the Docker sandbox, whose runtime image removes the agent's shell and file-write tools. | `LLM_API_KEY` or `OPENAI_API_KEY`<br><details><summary>Optional config</summary><code>DEFAULT_MODEL</code>, <code>DEFAULT_BASE_URL</code> (same defaults as <code>aig</code>), <code>DEFAULT_MODEL_CONTEXT_WINDOW</code>, <code>AIG_MCP_SCAN_ALLOW_HOST_EXEC=1</code> (required with <code>--sandbox off</code>).</details> | runtime image (patched `aig-mcp-scan==0.2.0`); `pip install aig-mcp-scan==0.2.0` is unpatched |
| `cisco` | Cisco AI Defense skill-scanner | [repo](https://github.com/cisco-ai-defense/skill-scanner) | Local file or directory scanner invoked through `skill-scanner` with JSON report output. Optional upstream env vars enable LLM, VirusTotal, and Cisco AI Defense analyzers. | none<br><details><summary>Optional config</summary><code>SKILL_SCANNER_LLM_API_KEY</code>, <code>SKILL_SCANNER_LLM_PROVIDER</code>, <code>SKILL_SCANNER_LLM_MODEL</code>, <code>SKILL_SCANNER_LLM_BASE_URL</code>, <code>SKILL_SCANNER_LLM_USER</code>, <code>SKILL_SCANNER_LLM_API_VERSION</code>, <code>SKILL_SCANNER_LLM_FORCE_JSON_OBJECT</code>, <code>SKILL_SCANNER_META_LLM_API_KEY</code>, <code>SKILL_SCANNER_META_LLM_MODEL</code>, <code>SKILL_SCANNER_META_LLM_BASE_URL</code>, <code>SKILL_SCANNER_META_LLM_API_VERSION</code>, <code>AWS_PROFILE</code>, <code>AWS_REGION</code>, <code>GOOGLE_APPLICATION_CREDENTIALS</code>, <code>VIRUSTOTAL_API_KEY</code>, <code>AI_DEFENSE_API_KEY</code>, <code>AI_DEFENSE_API_URL</code>.</details> | `uv pip install cisco-ai-skill-scanner` |
| `relyable` | Relyable | [repo](https://github.com/veriker/relyable) | Functional re-derivation evidence: does the skill still do what its docs claim, recomputed? Emits the strongest grade that applies. `exogenous`: a declared `rederive.json` property manifest (idempotence / round-trip), with both sides of the relation computed from the skill's own code and the result mutation-tested against vacuity. `self_spec`: re-runs the author's own committed oracle (shipped tests or documented I/O examples). `cold_golden`: when an LLM key is set, a code-blind model infers goldens from SKILL.md alone and abstains unless the docs pin exact behavior; divergences are reported as unconfirmed, never as accusations. `non_rederivable`: the honest floor, never a fabricated pass. Functional axis only; complements the security scanners and does not detect malware or prompt injection. Skill code runs only inside the Docker sandbox (or with an explicit opt-in), in a scrubbed environment, and the scanner fails closed otherwise. Not preinstalled in the `clawscan-runtime` image. | none<br><details><summary>Optional config</summary><code>RELYABLE_SCAN_ALLOW_HOST_EXEC</code> — explicit ack that the host is disposable when running with <code>--sandbox off</code>.<br><br><code>RELYABLE_LLM_API_KEY</code> (+ <code>RELYABLE_LLM_PROVIDER</code> <code>anthropic|openai</code>, <code>RELYABLE_LLM_MODEL</code>, <code>RELYABLE_LLM_BASE_URL</code>) — explicit per-scanner opt-in that enables the <code>cold_golden</code> lane; key presence only is ever recorded in the payload. Generic <code>ANTHROPIC_API_KEY</code>/<code>OPENAI_API_KEY</code> are honored by standalone <code>relyable-scan</code> but are deliberately not auto-forwarded by ClawScan.</details> | `clawscan install relyable` — not preinstalled in the runtime image |
| `skillspector` | NVIDIA SkillSpector | [repo](https://github.com/NVIDIA/skillspector) | Local skill or OpenClaw plugin file/directory scanner. Uses LLM mode when provider env vars are set; otherwise runs with `--no-llm`. | none<br><details><summary>Optional config</summary><code>SKILLSPECTOR_PROVIDER</code>, <code>SKILLSPECTOR_MODEL</code>, <code>SKILLSPECTOR_REASONING_EFFORT</code>, <code>SKILLSPECTOR_TEMPERATURE</code>, <code>SKILLSPECTOR_SEED</code>, <code>SKILLSPECTOR_MODEL_REGISTRY</code>, <code>SKILLSPECTOR_LOG_LEVEL</code>, <code>SKILLSPECTOR_SSL_VERIFY</code>, <code>NVIDIA_INFERENCE_KEY</code>, <code>OPENAI_API_KEY</code>, <code>OPENAI_BASE_URL</code>, <code>ANTHROPIC_API_KEY</code>, <code>ANTHROPIC_PROXY_ENDPOINT_URL</code>, <code>ANTHROPIC_PROXY_API_KEY</code>, <code>ANTHROPIC_PROXY_API_VERSION</code>.</details> | `uv tool install git+https://github.com/NVIDIA/skillspector.git` |
| `snyk` | Snyk Agent Scan | [repo](https://github.com/snyk/agent-scan) | Local skill scanner invoked through `uvx snyk-agent-scan`. | `SNYK_TOKEN` | verifies `uvx` launcher |
| `socket` | Socket CLI | [repo](https://github.com/SocketDev/socket-cli) | Local file or directory scanner using Socket's public CLI full-scan path. | `SOCKET_CLI_API_TOKEN` | `npm install -g socket` |
Starting in `v0.1.2`, the built-in `aig` scanner uses Tencent's local
`aig-skill-scan` package instead of the legacy A.I.G Docker/API service.
Replace `AIG_MODEL` with `DEFAULT_MODEL`, `AIG_MODEL_BASE_URL` with
`DEFAULT_BASE_URL`, and `AIG_MODEL_API_KEY` with `LLM_API_KEY` (or
`OPENAI_API_KEY`). `AIG_BASE_URL` and `AIG_API_KEY` are no longer used.
The local scanner accepts directory targets only; materialize URL or file inputs
as a skill directory before scanning them with `aig`.
## Profiles
`--profile` runs a saved scanner and judge configuration, such as the built-in
`clawhub` profile that matches ClawHub's production scanner suite and Codex
judge harness:
```bash
clawscan ./my-skill --profile clawhub
```
The same profile accepts an explicit OpenClaw plugin directory (or its
`openclaw.plugin.json` manifest). For plugins it runs only `aig-mcp`, Tencent
A.I.G's MCP code audit, over the whole plugin minus `node_modules` (embedded
skills and `dist/`/`build/` output included), and renders the bundled judge
prompt with `packageRelease` target context and that one raw SARIF result; see
[docs/scanners.md](docs/scanners.md#plugin-code-audit-with-aig-mcp).
Inspect the built-in profile catalog:
```bash
clawscan profiles
clawscan profiles -v
```
### Available profiles
| Profile | Scanners | Judge |
| --- | --- | --- |
| `clawhub` | skills: `skillspector`, `aig`; plugins: `aig-mcp` | OpenAI Decisions API `gpt-6-luna` verdict (`clawscan-decisions-judge`), then a Codex `gpt-6-luna` low-effort overview |
| `clawhub-aig` | `skillspector`, `aig` | Same judge and overview as `clawhub` |
| `openclaw-install-policy` | `skillspector` | none |
The two ClawHub profiles explicitly configure the OpenAI scanner lanes with
`scannerEnvDefaults`: SkillSpector uses `SKILLSPECTOR_PROVIDER=openai`,
`SKILLSPECTOR_MODEL=gpt-6.1-sol`, and `SKILLSPECTOR_REASONING_EFFORT=medium`;
A.I.G uses `DEFAULT_MODEL=gpt-6.1-sol` and `REASONING_EFFORT=medium` with OpenAI
credentials (`aig-mcp` uses the same model and endpoint defaults, but not
`REASONING_EFFORT` or the Standard-tier control). The verdict comes from the
OpenAI Decisions API (`gpt-6-luna`) through `clawscan-decisions-judge`, which returns typed answers only; a separate
Codex overview step (`gpt-6-luna`, low reasoning, Standard service) writes the
readable summary and never changes the verdict. Credentials stay in the
environment. See [judge harness](docs/judge.md#clawhub-decisions-judge) for the
request shape, verdict mapping, and environment variables.
Explicit model/provider/effort environment overrides win. A different
SkillSpector provider retains its own model defaults. A.I.G with a dedicated
`LLM_API_KEY` retains its provider defaults unless `DEFAULT_BASE_URL` explicitly
selects `https://api.openai.com/v1`; third-party endpoints are never redirected.
An explicit alternative model does not inherit the profile's Sol effort.
Generic `--scanner` runs and project profiles without `scannerEnvDefaults` keep
their existing provider choices. The supported defaults are only the five model
control names above, not arbitrary environment variables or credentials.
Use the runtime image built from the same source revision: its pinned NVIDIA
SkillSpector and Tencent Zhuque Lab A.I.G integrations include narrow Responses
API compatibility patches. Sol tool calls are not supported on Chat Completions;
changing only the model environment or reusing the old 0.2.0 runtime is insufficient.
See [scanner runtime compatibility](docs/scanner-runtime.md).
### Build a custom profile with `.clawscan.yml`
Custom profiles can be created in `.clawscan.yml`.
This is useful for version controlling iterations on your profile, creating multiple profiles to run over the same skills, etc
```yaml
# yaml-language-server: $schema=https://raw.githubusercontent.com/openclaw/clawscan/main/schemas/clawscan.schema.json
version: 1
profiles:
review:
scanners:
- skillspector
- snyk
sandbox:
env:
- OPENAI_API_KEY
- CODEX_API_KEY
judge:
command: >
codex exec --cd {{ workspace }}
--model gpt-5.5
--output-last-message {{ output }}
- < {{ prompt:./prompt.md }}
```
The published schema provides editor completion and catches invalid profile
fields before a scan starts. ClawScan also validates every config at runtime.
## Judge Harness
`--judge` hands scanner evidence to an external agent command so it can inspect
the skill, do its own research in the scan workspace, and write a final JSON
verdict:
```bash
clawscan ./my-skill \
--scanner skillspector \
--judge 'codex exec --cd {{ workspace }} --output-last-message {{ output }} - < {{ prompt:./prompt.md }}'
```
Supported `--judge` placeholders:
| Placeholder | Meaning |
| --- | --- |
| `{{ workspace }}` | Temporary directory containing the copied skill, scanner JSON, and metadata. |
| `{{ judge_sandbox }}` | `danger-full-access` inside ClawScan's Docker sandbox, otherwise `read-only`. |
| `{{ prompt }}` | Render `./prompt.md` and pass the rendered prompt file path. |
| `{{ prompt:<path> }}` | Render a specific prompt template and pass that file path. |
| `{{ output_schema }}` | Copy `./schema.json` into the workspace and pass that file path. |
| `{{ output_schema:<path> }}` | Copy a specific schema file and pass that file path. |
| `{{ output }}` | File path where the judge should write its final JSON object. |
The judge is any command that reads the prompt and writes a JSON object matching
the output schema. ClawScan is judge-agnostic: judge-specific checks, such as an
input-size limit, belong inside the judge command, and a failed judge command
keeps its own error. A profile can add an `overview.command` that writes a
readable summary after the verdict; it never changes the verdict or gate. See
[judge harness](docs/judge.md).
## Sandbox
ClawScan runs command-backed scanners and judges in
`ghcr.io/openclaw/clawscan-runtime:latest` by default:
```bash
clawscan ./my-skill --scanner skillspector
```
Use `--sandbox off` only in an already-isolated environment, or when you have
installed scanner dependencies on the host with `clawscan install`. Use
`--sandbox-env <NAME>` or a profile `sandbox.env` list to pass judge-specific
environment variables into the container.
## Benchmarks
`clawscan benchmark <benchmark-id>` runs a supported benchmark through the
selected scanners and optional judge harness:
```bash
clawscan benchmark list
clawscan benchmark SkillTrustBench \
--profile clawhub \
--output ./artifacts/skilltrustbench-clawhub.json
```
Use `--ids <path-or-url>` with SkillTrustBench to run a fixed subset from a
plain text ID list or JSONL rows with an `id` field.
### Available benchmarks
| Benchmark | ID | Source |
| --- | --- | --- |
| ClawHub Security Signals | `clawhub-security-signals` | [Hugging Face](https://huggingface.co/datasets/OpenClaw/clawhub-security-signals) |
| SkillTrustBench | `SkillTrustBench` | [Hugging Face](https://huggingface.co/datasets/cuhk-zhuque/SkillTrustBench) |
### Submitting a patch to the `clawhub` profile
If you are a security researcher who found malicious skills live on ClawHub and
want to improve the production scanner so it catches them, use GitHub private
vulnerability reporting for the sensitive details and open a PR containing only
a candidate `proposals/<GHSA-ID>/clawscan.yml` config. For a guided walkthrough,
ask Codex:
```text
Use $report-clawhub-malicious-skill to walk me through reporting a malicious ClawHub skill.
```
### ClawHub Profile Baseline
Maintainers validate accepted `clawhub` profile proposals against the public
SkillTrustBench leaderboard subset. The maintainer gate writes compact dated
baselines under `benchmarks/skilltrustbench-leaderboard-10pct/`; the latest
`YYYY-MM-DD.json` file is the current accepted baseline.