# ClawScan 📡
ClawScan is a composable security scanning harness for agent skills.
Run a suite of skill security scanners, pass the results to a judge harness, and compare against multiple skill security benchmarks.
[](https://github.com/openclaw/clawscan/actions/workflows/ci.yml?query=branch%3Amain)
[](https://github.com/openclaw/clawscan/actions/workflows/release.yml)
[](https://github.com/openclaw/clawscan/releases)
## Quick Start
Install ClawScan:
```bash
npm install -g @openclaw/clawscan
```
The macOS binaries support macOS 13 or later on Apple Silicon and Intel. CI uses
the Apple Silicon `xcode-27` runner and verifies x86_64 binaries under Rosetta.
Release and npm builds pin the macOS 13 deployment target and check both binaries' minimum OS metadata.
Source builds require Go 1.27.1 or later; the module selects the patched Go 1.27.3 build toolchain.
Command-backed scanners and judges run in ClawScan's Docker runtime by default,
so keep Docker running for local scans.
Run NVIDIA SkillSpector and Cisco Skill Scanner against a local `skills/` folder:
```bash
clawscan --scanner skillspector --scanner cisco
```
## Scan a known malicious skill
This example scans Trail of Bits' [`csv-summarizer`](https://github.com/trailofbits/overtly-malicious-skills/tree/4ffbf9461ef0505f9ce76a0d3694a18ec33ea531/skills/csv-summarizer) skill, which claims to summarize a CSV file but also prints every environment variable when run.
```bash
git clone https://github.com/trailofbits/overtly-malicious-skills.git /tmp/overtly-malicious-skills
cd /tmp/overtly-malicious-skills
git checkout 4ffbf9461ef0505f9ce76a0d3694a18ec33ea531
clawscan skills/csv-summarizer \
--scanner skillspector \
--scanner cisco \
--output /tmp/clawscan-csv-summarizer.json
```
Sample findings:
```txt
targets: 1
scanner_completed: 2
scanner_failed: 0
scanner_skipped: 0
issues_found: 2
gate: pass
errors: 0
full_results: /tmp/clawscan-csv-summarizer.json
```
The results bundle keeps the top-level artifact plus per-scanner JSON reports.
<details>
<summary>Artifact excerpt</summary>
```json
{
"schemaVersion": "clawscan-run-v1",
"target": "skills/csv-summarizer",
"scanners": {
"cisco": {
"status": "completed",
"durationMs": 42,
"outputPath": "clawscan-csv-summarizer/skills/csv-summarizer/cisco.json",
"isSafe": true,
"maxSeverity": "SAFE",
"findingsCount": 0
},
"skillspector": {
"status": "completed",
"durationMs": 42,
"outputPath": "clawscan-csv-summarizer/skills/csv-summarizer/skillspector.json",
"severity": "MEDIUM",
"score": 31,
"recommendation": "CAUTION",
"issues": [
{
"id": "LP3",
"severity": "MEDIUM",
"file": "SKILL.md"
},
{
"id": "E2",
"severity": "HIGH",
"file": "scripts/summarize.py"
}
]
}
}
}
```
</details>
## Motivation
Agent-skill security is new and fast-moving, with researchers and companies
exploring many promising scanners, datasets, and judge harnesses. In our
[ClawHub Security Signals paper](https://arxiv.org/html/2606.01494v1), we found
that combining multiple scanners with a configurable judge works better than
relying on any single scanner.
ClawScan turns that approach into a repeatable CLI. It includes a built-in `clawhub` profile, a saved scanner-and-judge configuration that matches what ClawHub runs in production, so researchers can reproduce results, test improvements, and help improve detection against the weekly refreshed ClawHub security-signals dataset.
## Commands
| Command family | Use |
| --- | --- |
| `clawscan <target> --scanner <id>` | Run one or more scanners against an explicit skill or OpenClaw plugin target. Omit `<target>` to scan child skill directories under `./skills`; plugins are never auto-discovered. |
| `clawscan scanners [list\|<scanner-id>]` | Discover supported scanner IDs, required env vars, upstream links, descriptions, and install guidance. |
| `clawscan profiles [-v]` | Inspect built-in profiles; `-v` prints the catalog as YAML. |
| `clawscan benchmark [list\|<benchmark-id>]` | Discover or run supported benchmarks through a selected scanner/profile/judge setup. |
| `clawscan install <scanner-id> [...]` | Install or verify local scanner dependencies where ClawScan has registry-backed install plans. |
| `clawscan openclaw-install-policy` | Act as an external OpenClaw `security.installPolicy.exec` command. Reads the staged install request from stdin and returns allow/warn/block JSON. |
## OpenClaw install policy
ClawScan integrates with OpenClaw at the operator-owned
`security.installPolicy` boundary. It does not register an install hook or
depend on plugin activation. The policy command scans the staged `sourcePath`
for both skills and plugins before OpenClaw commits a supported install or
update.
See [docs/openclaw-install-policy.md](docs/openclaw-install-policy.md) for the
trusted executable setup, configuration, payload contract, and scope.
## Scanners
`--scanner` selects a scanner adapter to run, writes its raw JSON evidence into
the results artifact, and can be repeated to compare multiple scanners in one
run:
```bash
clawscan ./my-skill \
--scanner skillspector \
--scanner cisco
```
Discover the scanner catalog from the CLI:
```bash
clawscan scanners
clawscan scanners skillspector
```
### Available scanners
> **Want to add your scanner to the list?** Follow the guide in [docs/scanners.md](docs/scanners.md#adding-a-built-in-scanner-adapter)
| ID | Name | Repo | Description | Required env vars | Local dependency setup |
| --- | --- | --- | --- | --- | --- |
| `agentverus` | AgentVerus | [repo](https://github.com/agentverus/agentverus-scanner) | Local file or directory scanner invoked through agentverus-scanner. | none | `npm install --save-dev agentverus-scanner` |
| `aig` | Tencent AI-Infra-Guard | [repo](https://github.com/Tencent/AI-Infra-Guard/tree/main/skill-scan) | Tencent Zhuque Lab's local directory scanner invoked through `aig-skill-scan`. Produces SARIF 2.1.0 with SkillTrustBench T01-T09 evidence. | `LLM_API_KEY` or `OPENAI_API_KEY`<br><details><summary>Optional config</summary><code>DEFAULT_MODEL</code>, <code>REASONING_EFFORT</code> (requires the patched runtime), <code>DEFAULT_BASE_URL</code>, <code>DEFAULT_MODEL_CONTEXT_WINDOW</code>, <code>LOG_LEVEL</code>.</details> | `pip install aig-skill-scan` |
| `cisco` | Cisco AI Defense skill-scanner | [repo](https://github.com/cisco-ai-defense/skill-scanner) | Local file or directory scanner invoked through `skill-scanner` with JSON report output. Optional upstream env vars enable LLM, VirusTotal, and Cisco AI Defense analyzers. | none<br><details><summary>Optional config</summary><code>SKILL_SCANNER_LLM_API_KEY</code>, <code>SKILL_SCANNER_LLM_PROVIDER</code>, <code>SKILL_SCANNER_LLM_MODEL</code>, <code>SKILL_SCANNER_LLM_BASE_URL</code>, <code>SKILL_SCANNER_LLM_USER</code>, <code>SKILL_SCANNER_LLM_API_VERSION</code>, <code>SKILL_SCANNER_LLM_FORCE_JSON_OBJECT</code>, <code>SKILL_SCANNER_META_LLM_API_KEY</code>, <code>SKILL_SCANNER_META_LLM_MODEL</code>, <code>SKILL_SCANNER_META_LLM_BASE_URL</code>, <code>SKILL_SCANNER_META_LLM_API_VERSION</code>, <code>AWS_PROFILE</code>, <code>AWS_REGION</code>, <code>GOOGLE_APPLICATION_CREDENTIALS</code>, <code>VIRUSTOTAL_API_KEY</code>, <code>AI_DEFENSE_API_KEY</code>, <code>AI_DEFENSE_API_URL</code>.</details> | `uv pip install cisco-ai-skill-scanner` |
| `clawscan-static` | ClawScan Static | [repo](https://github.com/openclaw/clawscan) | Built-in deterministic scanner for high-signal risky skill and OpenClaw plugin patterns; packaged Python bytecode and NUL-obfuscated text are flagged and inspected, while opaque binary omissions remain visible as low-severity evidence. | none | skipped; built in |
| `relyable` | Relyable | [repo](https://github.com/veriker/relyable) | Functional re-derivation evidence: does the skill still do what its docs claim, recomputed? Emits the strongest grade that applies. `exogenous`: a declared `rederive.json` property manifest (idempotence / round-trip), with both sides of the relation computed from the skill's own code and the result mutation-tested against vacuity. `self_spec`: re-runs the author's own committed oracle (shipped tests or documented I/O examples). `cold_golden`: when an LLM key is set, a code-blind model infers goldens from SKILL.md alone and abstains unless the docs pin exact behavior; divergences are reported as unconfirmed, never as accusations. `non_rederivable`: the honest floor, never a fabricated pass. Functional axis only; complements the security scanners and does not detect malware or prompt injection. Skill code runs only inside the Docker sandbox (or with an explicit opt-in), in a scrubbed environment, and the scanner fails closed otherwise. Not preinstalled in the `clawscan-runtime` image. | none<br><details><summary>Optional config</summary><code>RELYABLE_SCAN_ALLOW_HOST_EXEC</code> — explicit ack that the host is disposable when running with <code>--sandbox off</code>.<br><br><code>RELYABLE_LLM_API_KEY</code> (+ <code>RELYABLE_LLM_PROVIDER</code> <code>anthropic|openai</code>, <code>RELYABLE_LLM_MODEL</code>, <code>RELYABLE_LLM_BASE_URL</code>) — explicit per-scanner opt-in that enables the <code>cold_golden</code> lane; key presence only is ever recorded in the payload. Generic <code>ANTHROPIC_API_KEY</code>/<code>OPENAI_API_KEY</code> are honored by standalone <code>relyable-scan</code> but are deliberately not auto-forwarded by ClawScan.</details> | `clawscan install relyable` — not preinstalled in the runtime image |
| `skillspector` | NVIDIA SkillSpector | [repo](https://github.com/NVIDIA/skillspector) | Local skill or OpenClaw plugin file/directory scanner. Uses LLM mode when provider env vars are set; otherwise runs with `--no-llm`. | none<br><details><summary>Optional config</summary><code>SKILLSPECTOR_PROVIDER</code>, <code>SKILLSPECTOR_MODEL</code>, <code>SKILLSPECTOR_REASONING_EFFORT</code>, <code>SKILLSPECTOR_TEMPERATURE</code>, <code>SKILLSPECTOR_SEED</code>, <code>SKILLSPECTOR_MODEL_REGISTRY</code>, <code>SKILLSPECTOR_LOG_LEVEL</code>, <code>SKILLSPECTOR_SSL_VERIFY</code>, <code>NVIDIA_INFERENCE_KEY</code>, <code>OPENAI_API_KEY</code>, <code>OPENAI_BASE_URL</code>, <code>ANTHROPIC_API_KEY</code>, <code>ANTHROPIC_PROXY_ENDPOINT_URL</code>, <code>ANTHROPIC_PROXY_API_KEY</code>, <code>ANTHROPIC_PROXY_API_VERSION</code>.</details> | `uv tool install git+https://github.com/NVIDIA/skillspector.git` |
| `snyk` | Snyk Agent Scan | [repo](https://github.com/snyk/agent-scan) | Local skill scanner invoked through `uvx snyk-agent-scan`. | `SNYK_TOKEN` | verifies `uvx` launcher |
| `socket` | Socket CLI | [repo](https://github.com/SocketDev/socket-cli) | Local file or directory scanner using Socket's public CLI full-scan path. | `SOCKET_CLI_API_TOKEN` | `npm install -g socket` |
| `virustotal` | VirusTotal API | [docs](https://docs.virustotal.com/reference/file) | API-backed local file hash lookup. Skill and OpenClaw plugin directories are scanned as deterministic ZIP archives. | `VIRUSTOTAL_API_KEY` | skipped; API-backed |
Starting in `v0.1.2`, the built-in `aig` scanner uses Tencent's local
`aig-skill-scan` package instead of the legacy A.I.G Docker/API service.
Replace `AIG_MODEL` with `DEFAULT_MODEL`, `AIG_MODEL_BASE_URL` with
`DEFAULT_BASE_URL`, and `AIG_MODEL_API_KEY` with `LLM_API_KEY` (or
`OPENAI_API_KEY`). `AIG_BASE_URL` and `AIG_API_KEY` are no longer used.
The local scanner accepts directory targets only; materialize URL or file inputs
as a skill directory before scanning them with `aig`.
## Profiles
`--profile` runs a saved scanner and judge configuration, such as the built-in
`clawhub` profile that matches ClawHub's production scanner suite and Codex
judge harness:
```bash
clawscan ./my-skill --profile clawhub
```
The same profile accepts an explicit OpenClaw plugin directory (or its
`openclaw.plugin.json` manifest), runs the plugin-capable scanners in the
profile, and renders the bundled judge prompt with `packageRelease` target
context.
Inspect the built-in profile catalog:
```bash
clawscan profiles
clawscan profiles -v
```
### Available profiles
| Profile | Scanners | Judge |
| --- | --- | --- |
| `clawhub` | `skillspector`, `clawscan-static`, `aig` | Codex `gpt-6.1-sol`, high reasoning, Standard service, bundled ClawHub prompt/schema |
| `clawhub-aig` | `skillspector`, `aig` | Codex `gpt-6.1-sol`, high reasoning, Standard service, bundled ClawHub prompt/schema |
| `openclaw-install-policy` | `skillspector`, `clawscan-static` | none |
The two ClawHub profiles explicitly configure the OpenAI scanner lanes with
`scannerEnvDefaults`: SkillSpector uses `SKILLSPECTOR_PROVIDER=openai`,
`SKILLSPECTOR_MODEL=gpt-6.1-sol`, and `SKILLSPECTOR_REASONING_EFFORT=medium`;
A.I.G uses `DEFAULT_MODEL=gpt-6.1-sol` and `REASONING_EFFORT=medium` with OpenAI
credentials. The final Codex judge uses `gpt-6.1-sol`, high reasoning, and default
Standard service (no fast/priority override). Credentials stay in the environment.
Explicit model/provider/effort environment overrides win. A different
SkillSpector provider retains its own model defaults. A.I.G with a dedicated
`LLM_API_KEY` retains its provider defaults unless `DEFAULT_BASE_URL` explicitly
selects `https://api.openai.com/v1`; third-party endpoints are never redirected.
An explicit alternative model does not inherit the profile's Sol effort.
Generic `--scanner` runs and project profiles without `scannerEnvDefaults` keep
their existing provider choices. The supported defaults are only the five model
control names above, not arbitrary environment variables or credentials.
Use the runtime image built from the same source revision: its pinned NVIDIA
SkillSpector and Tencent Zhuque Lab A.I.G integrations include narrow Responses
API compatibility patches. Sol tool calls are not supported on Chat Completions;
changing only the model environment or reusing the old 0.2.0 runtime is insufficient.
See [scanner runtime compatibility](docs/scanner-runtime.md).
### Build a custom profile with `.clawscan.yml`
Custom profiles can be created in `.clawscan.yml`.
This is useful for version controlling iterations on your profile, creating multiple profiles to run over the same skills, etc
```yaml
# yaml-language-server: $schema=https://raw.githubusercontent.com/openclaw/clawscan/main/schemas/clawscan.schema.json
version: 1
profiles:
review:
scanners:
- skillspector
- snyk
sandbox:
env:
- OPENAI_API_KEY
- CODEX_API_KEY
judge:
command: >
codex exec --cd {{ workspace }}
--model gpt-5.5
--output-last-message {{ output }}
- < {{ prompt:./prompt.md }}
```
The published schema provides editor completion and catches invalid profile
fields before a scan starts. ClawScan also validates every config at runtime.
## Judge Harness
`--judge` hands scanner evidence to an external agent command so it can inspect
the skill, do its own research in the scan workspace, and write a final JSON
verdict:
```bash
clawscan ./my-skill \
--scanner skillspector \
--judge 'codex exec --cd {{ workspace }} --output-last-message {{ output }} - < {{ prompt:./prompt.md }}'
```
Supported `--judge` placeholders:
| Placeholder | Meaning |
| --- | --- |
| `{{ workspace }}` | Temporary directory containing the copied skill, scanner JSON, and metadata. |
| `{{ judge_sandbox }}` | `danger-full-access` inside ClawScan's Docker sandbox, otherwise `read-only`. |
| `{{ prompt }}` | Render `./prompt.md` and pass the rendered prompt file path. |
| `{{ prompt:<path> }}` | Render a specific prompt template and pass that file path. |
| `{{ output_schema }}` | Copy `./schema.json` into the workspace and pass that file path. |
| `{{ output_schema:<path> }}` | Copy a specific schema file and pass that file path. |
| `{{ output }}` | File path where the judge should write its final JSON object. |
## Sandbox
ClawScan runs command-backed scanners and judges in
`ghcr.io/openclaw/clawscan-runtime:latest` by default:
```bash
clawscan ./my-skill --scanner skillspector
```
Use `--sandbox off` only in an already-isolated environment, or when you have
installed scanner dependencies on the host with `clawscan install`. Use
`--sandbox-env <NAME>` or a profile `sandbox.env` list to pass judge-specific
environment variables into the container.
## Benchmarks
`clawscan benchmark <benchmark-id>` runs a supported benchmark through the
selected scanners and optional judge harness:
```bash
clawscan benchmark list
clawscan benchmark SkillTrustBench \
--profile clawhub \
--output ./artifacts/skilltrustbench-clawhub.json
```
Use `--ids <path-or-url>` with SkillTrustBench to run a fixed subset from a
plain text ID list or JSONL rows with an `id` field.
### Available benchmarks
| Benchmark | ID | Source |
| --- | --- | --- |
| ClawHub Security Signals | `clawhub-security-signals` | [Hugging Face](https://huggingface.co/datasets/OpenClaw/clawhub-security-signals) |
| SkillTrustBench | `SkillTrustBench` | [Hugging Face](https://huggingface.co/datasets/cuhk-zhuque/SkillTrustBench) |
### Submitting a patch to the `clawhub` profile
If you are a security researcher who found malicious skills live on ClawHub and
want to improve the production scanner so it catches them, use GitHub private
vulnerability reporting for the sensitive details and open a PR containing only
a candidate `proposals/<GHSA-ID>/clawscan.yml` config. For a guided walkthrough,
ask Codex:
```text
Use $report-clawhub-malicious-skill to walk me through reporting a malicious ClawHub skill.
```
### ClawHub Profile Baseline
Maintainers validate accepted `clawhub` profile proposals against the public
SkillTrustBench leaderboard subset. The maintainer gate writes compact dated
baselines under `benchmarks/skilltrustbench-leaderboard-10pct/`; the latest
`YYYY-MM-DD.json` file is the current accepted baseline.