ClawScan 📡

ClawScan is a composable security scanning harness for agent skills.

Run a suite of skill security scanners, pass the results to a judge harness, and compare against multiple skill security benchmarks.

Quick Start

Install ClawScan:

npm install -g @openclaw/clawscan

Command-backed scanners and judges run in ClawScan's Docker runtime by default, so keep Docker running for local scans.

Run NVIDIA SkillSpector and Cisco Skill Scanner against a local skills/ folder:

clawscan --scanner skillspector --scanner cisco

Scan a known malicious skill

This example scans Trail of Bits' csv-summarizer skill, which claims to summarize a CSV file but also prints every environment variable when run.

git clone https://github.com/trailofbits/overtly-malicious-skills.git /tmp/overtly-malicious-skills
cd /tmp/overtly-malicious-skills
git checkout 4ffbf9461ef0505f9ce76a0d3694a18ec33ea531
clawscan skills/csv-summarizer \
  --scanner skillspector \
  --scanner cisco \
  --output /tmp/clawscan-csv-summarizer.json

Sample findings:

targets: 1
scanner_completed: 2
scanner_failed: 0
scanner_skipped: 0
issues_found: 2
errors: 0
full_results: /tmp/clawscan-csv-summarizer.json

The results bundle keeps the top-level artifact plus per-scanner JSON reports.

{
  "schemaVersion": "clawscan-run-v1",
  "target": "skills/csv-summarizer",
  "scanners": {
    "cisco": {
      "status": "completed",
      "durationMs": 42,
      "outputPath": "clawscan-csv-summarizer/skills/csv-summarizer/cisco.json",
      "isSafe": true,
      "maxSeverity": "SAFE",
      "findingsCount": 0
    },
    "skillspector": {
      "status": "completed",
      "durationMs": 42,
      "outputPath": "clawscan-csv-summarizer/skills/csv-summarizer/skillspector.json",
      "severity": "MEDIUM",
      "score": 31,
      "recommendation": "CAUTION",
      "issues": [
        {
          "id": "LP3",
          "severity": "MEDIUM",
          "file": "SKILL.md"
        },
        {
          "id": "E2",
          "severity": "HIGH",
          "file": "scripts/summarize.py"
        }
      ]
    }
  }
}

Motivation

Agent-skill security is new and fast-moving, with researchers and companies exploring many promising scanners, datasets, and judge harnesses. In our ClawHub Security Signals paper, we found that combining multiple scanners with a configurable judge works better than relying on any single scanner.

ClawScan turns that approach into a repeatable CLI. It includes a built-in clawhub profile, a saved scanner-and-judge configuration that matches what ClawHub runs in production, so researchers can reproduce results, test improvements, and help improve detection against the weekly refreshed ClawHub security-signals dataset.

Commands

| Command family | Use | | --- | --- | | clawscan <target> --scanner <id> | Run one or more scanners against an explicit target. Omit <target> to scan child skill directories under ./skills. | | clawscan scanners [list\|<scanner-id>] | Discover supported scanner IDs, required env vars, upstream links, descriptions, and install guidance. | | clawscan profiles [-v] | Inspect built-in plus nearest project-local profiles; -v prints the resolved profile catalog as YAML. | | clawscan benchmark [list\|<benchmark-id>] | Discover or run supported benchmarks through a selected scanner/profile/judge setup. | | clawscan install <scanner-id> [...] | Install or verify local scanner dependencies where ClawScan has registry-backed install plans. |

Scanners

--scanner selects a scanner adapter to run, writes its raw JSON evidence into the results artifact, and can be repeated to compare multiple scanners in one run:

clawscan ./my-skill \
  --scanner skillspector \
  --scanner cisco

Discover the scanner catalog from the CLI:

clawscan scanners
clawscan scanners skillspector

Available scanners

Want to add your scanner to the list? Follow the guide in docs/scanners.md

| ID | Name | Repo | Description | Required env vars | Local dependency setup | | --- | --- | --- | --- | --- | --- | | agentverus | AgentVerus | repo | Local file or directory scanner invoked through agentverus-scanner. | none | npm install --save-dev agentverus-scanner | | aig | Tencent AI-Infra-Guard | repo | Tencent Zhuque Lab's local directory scanner invoked through aig-skill-scan. Produces SARIF 2.1.0 with SkillTrustBench T01-T09 evidence. | LLM_API_KEY or OPENAI_API_KEYOptional configDEFAULT_MODEL, DEFAULT_BASE_URL, DEFAULT_MODEL_CONTEXT_WINDOW, LOG_LEVEL. | pip install aig-skill-scan | | cisco | Cisco AI Defense skill-scanner | repo | Local file or directory scanner invoked through skill-scanner with JSON report output. Optional upstream env vars enable LLM, VirusTotal, and Cisco AI Defense analyzers. | noneOptional configSKILL_SCANNER_LLM_API_KEY, SKILL_SCANNER_LLM_PROVIDER, SKILL_SCANNER_LLM_MODEL, SKILL_SCANNER_LLM_BASE_URL, SKILL_SCANNER_LLM_USER, SKILL_SCANNER_LLM_API_VERSION, SKILL_SCANNER_LLM_FORCE_JSON_OBJECT, SKILL_SCANNER_META_LLM_API_KEY, SKILL_SCANNER_META_LLM_MODEL, SKILL_SCANNER_META_LLM_BASE_URL, SKILL_SCANNER_META_LLM_API_VERSION, AWS_PROFILE, AWS_REGION, GOOGLE_APPLICATION_CREDENTIALS, VIRUSTOTAL_API_KEY, AI_DEFENSE_API_KEY, AI_DEFENSE_API_URL. | uv pip install cisco-ai-skill-scanner | | clawscan-static | ClawScan Static | repo | Built-in deterministic text scanner for high-signal risky skill patterns. | none | skipped; built in | | skillspector | NVIDIA SkillSpector | repo | Local file or directory scanner. Uses LLM mode when provider env vars are set; otherwise runs with --no-llm. | noneOptional configSKILLSPECTOR_PROVIDER, SKILLSPECTOR_MODEL, SKILLSPECTOR_MODEL_REGISTRY, SKILLSPECTOR_LOG_LEVEL, SKILLSPECTOR_SSL_VERIFY, NVIDIA_INFERENCE_KEY, OPENAI_API_KEY, OPENAI_BASE_URL, ANTHROPIC_API_KEY, ANTHROPIC_PROXY_ENDPOINT_URL, ANTHROPIC_PROXY_API_KEY, ANTHROPIC_PROXY_API_VERSION. | uv tool install git+https://github.com/NVIDIA/skillspector.git | | snyk | Snyk Agent Scan | repo | Local skill scanner invoked through uvx snyk-agent-scan. | SNYK_TOKEN | verifies uvx launcher | | socket | Socket CLI | repo | Local file or directory scanner using Socket's public CLI full-scan path. | SOCKET_CLI_API_TOKEN | npm install -g socket | | virustotal | VirusTotal API | docs | API-backed single local file hash lookup. Directories return a skipped result. | VIRUSTOTAL_API_KEY | skipped; API-backed |

Starting in v0.1.2, the built-in aig scanner uses Tencent's local aig-skill-scan package instead of the legacy A.I.G Docker/API service. Replace AIG_MODEL with DEFAULT_MODEL, AIG_MODEL_BASE_URL with DEFAULT_BASE_URL, and AIG_MODEL_API_KEY with LLM_API_KEY (or OPENAI_API_KEY). AIG_BASE_URL and AIG_API_KEY are no longer used. The local scanner accepts directory targets only; materialize URL or file inputs as a skill directory before scanning them with aig.

Profiles

--profile runs a saved scanner and judge configuration, such as the built-in clawhub profile that matches ClawHub's production scanner suite and Codex judge harness:

clawscan ./my-skill --profile clawhub

Inspect the resolved profile catalog, including the nearest project .clawscan.yml / .clawscan.yaml when present:

clawscan profiles
clawscan profiles -v

Available profiles

| Profile | Scanners | Judge | | --- | --- | --- | | clawhub | skillspector, clawscan-static | Codex gpt-5.5, high reasoning, bundled ClawHub prompt/schema |

Build a custom profile with .clawscan.yml

Custom profiles can be created in .clawscan.yml.

This is useful for version controlling iterations on your profile, creating multiple profiles to run over the same skills, etc

version: 1
profiles:
  review:
    scanners:
      - skillspector
      - snyk
    sandbox:
      env:
        - OPENAI_API_KEY
        - CODEX_API_KEY
    judge:
      command: >
        codex exec --cd {{ workspace }}
        --model gpt-5.5
        --output-last-message {{ output }}
        - < {{ prompt:./prompt.md }}

Judge Harness

--judge hands scanner evidence to an external agent command so it can inspect the skill, do its own research in the scan workspace, and write a final JSON verdict:

clawscan ./my-skill \
  --scanner skillspector \
  --judge 'codex exec --cd {{ workspace }} --output-last-message {{ output }} - < {{ prompt:./prompt.md }}'

Supported --judge placeholders:

| Placeholder | Meaning | | --- | --- | | {{ workspace }} | Temporary directory containing the copied skill, scanner JSON, and metadata. | | {{ judge_sandbox }} | danger-full-access inside ClawScan's Docker sandbox, otherwise read-only. | | {{ prompt }} | Render ./prompt.md and pass the rendered prompt file path. | | {{ prompt:<path> }} | Render a specific prompt template and pass that file path. | | {{ output_schema }} | Copy ./schema.json into the workspace and pass that file path. | | {{ output_schema:<path> }} | Copy a specific schema file and pass that file path. | | {{ output }} | File path where the judge should write its final JSON object. |

Sandbox

ClawScan runs command-backed scanners and judges in ghcr.io/openclaw/clawscan-runtime:latest by default:

clawscan ./my-skill --scanner skillspector

Use --sandbox off only in an already-isolated environment, or when you have installed scanner dependencies on the host with clawscan install. Use --sandbox-env <NAME> or a profile sandbox.env list to pass judge-specific environment variables into the container.

Benchmarks

clawscan benchmark <benchmark-id> runs a supported benchmark through the selected scanners and optional judge harness:

clawscan benchmark list

clawscan benchmark SkillTrustBench \
  --profile clawhub \
  --output ./artifacts/skilltrustbench-clawhub.json

Use --ids <path-or-url> with SkillTrustBench to run a fixed subset from a plain text ID list or JSONL rows with an id field.

Available benchmarks

| Benchmark | ID | Source | | --- | --- | --- | | ClawHub Security Signals | clawhub-security-signals | Hugging Face | | SkillTrustBench | SkillTrustBench | Hugging Face |

Submitting a patch to the clawhub profile

If you are a security researcher who found malicious skills live on ClawHub and want to improve the production scanner so it catches them, use GitHub private vulnerability reporting for the sensitive details and open a PR containing only a candidate proposals/<GHSA-ID>/clawscan.yml config. For a guided walkthrough, ask Codex:

Use $report-clawhub-malicious-skill to walk me through reporting a malicious ClawHub skill.

ClawHub Profile Baseline

Maintainers validate accepted clawhub profile proposals against the public SkillTrustBench leaderboard subset. The maintainer gate writes compact dated baselines under benchmarks/skilltrustbench-leaderboard-10pct/; the latest YYYY-MM-DD.json file is the current accepted baseline.