🔍 agent-pentest

Red team your AI agents before deployment.

One command. 41 attack vectors. Instant safety score.

npx agent-pentest scan --url https://your-agent.api.com

What It Does

Runs 41 automated adversarial tests against any AI agent endpoint in 4 categories:

| Category | Vectors | What It Tests | |---|---|---| | 💉 Prompt Injection | 11 | DAN, instruction override, delimiter bypass, CoT hijack | | 📤 Data Exfiltration | 10 | System prompt leak, API key extraction, env probing | | 🔓 Jailbreak | 10 | Roleplay, emotional manipulation, translation bypass | | 🛡️ Safety Bypass | 10 | Harmful content, medical misinfo, fraud, CSAM |

Returns a Safety Score (A-F) with detailed vulnerability report.

Quick Start

# Scan an agent (POST endpoint with JSON body)
npx agent-pentest scan --url https://my-agent.api.com/chat

# Custom body template
npx agent-pentest scan --url https://my-agent.api.com/chat \
  --body-template '{"prompt": "{{PAYLOAD}}", "max_tokens": 500}'

# Custom headers
npx agent-pentest scan --url https://my-agent.api.com/chat \
  -H "Authorization:Bearer sk-xxx" -H "X-Api-Key:my-key"

# Save report as markdown
npx agent-pentest scan --url https://my-agent.api.com/chat \
  --save report.md

# JSON output for CI/CD
npx agent-pentest scan --url https://my-agent.api.com/chat \
  --output json

# Fail CI if grade below B
npx agent-pentest scan --url https://my-agent.api.com/chat \
  --fail-under B

Commands

scan — Run a security scan

| Flag | Description | Default | |---|---|---| | -u, --url <url> | Target agent endpoint (required) | — | | -m, --method | HTTP method (POST/GET) | POST | | -H, --header | Custom headers (Key:Value) | — | | -b, --body-template | Body with {{PAYLOAD}} placeholder | {"message": "..."} | | -t, --timeout | Request timeout (ms) | 30000 | | -c, --concurrency | Parallel requests | 3 | | -o, --output | Format: terminal, json, markdown | terminal | | --save <path> | Save report to file | — | | --fail-under <grade> | Exit code 1 if below grade | — | | --categories | Filter vector categories | all |

vectors — List all attack vectors

npx agent-pentest vectors
npx agent-pentest vectors --category prompt-injection

CI/CD Integration

GitHub Action

- name: Agent Safety Scan
  run: npx agent-pentest scan --url ${{ secrets.AGENT_URL }} --fail-under B --output json --save safety-report.json

Safety Score

| Grade | Score | Meaning | |---|---|---| | A | 90-100 | Excellent — resistant to all tested vectors | | B | 80-89 | Good — minor warnings, no critical failures | | C | 70-79 | Fair — some vulnerabilities detected | | D | 50-69 | Poor — significant vulnerabilities | | F | 0-49 | Critical — agent is highly vulnerable |

PoE Receipt

Every scan generates a signed Proof of Execution receipt:

• SHA-256 hash of all results
• Timestamped signature for compliance audit trails
• Protocol: agent-pentest-v1

License

MIT — Berlin AI Labs