safetynet

A neural secret scanner that uses a fine-tuned ModernBERT model to detect leaked credentials in source code. Replaces regex-based tools like gitleaks with ML-powered NER (Named Entity Recognition) that understands context.

Ships with a pre-trained ONNX model — no training or model download required to get started.

Quick start

git clone <repo-url> safetynet && cd safetynet
npm install
npm run build

# Scan a directory
node dist/index.js detect --source /path/to/your/project

# Scan git history
node dist/index.js git --source /path/to/repo

# Pipe content via stdin
cat suspicious-file.py | node dist/index.js stdin

How it works

safetynet tokenizes your source code, runs it through a fine-tuned ModernBERT NER model, and produces BIO-tagged entity spans that identify secrets with character-level precision.

source code → fragment reader → NER classifier → span extractor → finding processor → reporter

The bundled model (models/) detects these entity types:

| Rule ID | Description | |---------|-------------| | credential-aws | AWS access keys and secret keys | | credential-api-key | API keys (sk-, api_, etc.) | | credential-token | Bearer tokens, JWTs, personal access tokens | | credential-password | Hardcoded password strings | | credential-connection-string | Database URLs, connection strings | | credential-private-key | PEM private key blocks | | credential-generic | Generic secrets not matching a specific type | | injection | SQL/code injection patterns | | escalation | Privilege escalation patterns |

Commands

detect — scan a directory

node dist/index.js detect --source /path/to/project
node dist/index.js detect --source . --report-format sarif

git — scan git history

node dist/index.js git --source /path/to/repo
node dist/index.js git --source . --commits abc123..def456

protect — pre-commit hook

# In .git/hooks/pre-commit:
node /path/to/safetynet/dist/index.js protect --source .

stdin — pipe content

echo 'AWS_SECRET_ACCESS_KEY = "wJalrXUtnFEMI..."' | node dist/index.js stdin

Options

| Flag | Default | Description | |------|---------|-------------| | --source <path> | . | Path to scan | | --config <path> | — | Path to .safetynet.toml | | --report-format | json | Output format: json, sarif, csv, junit | | --model <id> | safetynet-ner | Override model ID | | --model-dir <path> | — | Override model directory | | --fail-open | false | Exit 0 on classifier error (default: exit 2) | | --min-confidence | 0.5 | Suppress findings below this confidence | | --threshold | 0.85 | Below this, tag as low-confidence | | --concurrency | 4 | Worker pool concurrency | | --commits <sha> | — | Specific commit(s) to scan (git mode) |

Exit codes

| Code | Meaning | |------|---------| | 0 | No findings (clean) | | 1 | Findings detected | | 2 | Classifier error (use --fail-open to change to 0) |

Output formats

JSON (default) — gitleaks-compatible format:

[
  {
    "ruleId": "credential-aws",
    "description": "B-CREDENTIAL-AWS detected",
    "file": "config.py",
    "startLine": 12,
    "match": "AKIAIOSFODNN7EXAMPLE",
    "confidence": 0.92,
    "fingerprint": "abc123:config.py:credential-aws:12"
  }
]

SARIF — for GitHub Code Scanning, Azure DevOps, etc.

CSV — spreadsheet-friendly output.

JUnit — for CI test reporting.

Configuration

Create a .safetynet.toml in your project root:

[classifier]
model = "safetynet-ner"
threshold = 0.85
min_confidence = 0.5
concurrency = 4

[allowlist]
paths = ['^vendor/', '\\.test\\.ts$']
commits = ["abc123"]
stopwords = ["example", "test"]

Ignoring findings

.safetynetignore / .gitleaksignore

Add one fingerprint per line. .safetynetignore takes precedence:

abc123:src/config.ts:credential-aws:10
def456:src/db.ts:credential-password:5

Inline comments

api_key = "sk-test-key"  # safetynet:allow
aws_key = "AKIA..."      # gitleaks:allow

Both safetynet:allow and gitleaks:allow are recognized, making migration from gitleaks seamless.

Retraining the model

If you want to fine-tune on your own data:

cd training
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# Generate synthetic training data (or provide your own)
python3 data/generate_synthetic.py --output data/ner_dataset.json

# Fine-tune ModernBERT-base
python3 finetune_ner.py --data data/ner_dataset.json --epochs 3

# Export to ONNX with INT8 quantization
python3 export_onnx.py --model models/safetynet-ner --quantize

# Copy the quantized model to the bundled location
cp models/safetynet-ner-onnx/onnx/model_quantized.onnx ../models/onnx/
cp models/safetynet-ner-onnx/config.json ../models/
cp models/safetynet-ner-onnx/tokenizer.json ../models/
cp models/safetynet-ner-onnx/tokenizer_config.json ../models/

Development

npm run build        # Compile TypeScript
npm test             # Run all 174 tests
npm run lint         # Type check
npm run test:watch   # Watch mode

License

MIT