ai-authorship

Scan your git history to see how much of your code your AI tools actually wrote, which models did the work, and where those models have known weaknesses.

npx ai-authorship scan

Previously known as @mattersec/vibecheck. That package is deprecated — point your scripts at ai-authorship instead.

Why this exists

Almost every developer uses AI to write code. Almost nobody knows where their AI gets things wrong.

ai-authorship answers four questions about a repository:

1. How much of the code is AI-written? Authorship percentage across commits and contributors.
2. Which tools and models? Claude Code, Cursor, Copilot, Codex, and others, with confidence levels.
3. Where are the blind spots? Categories and languages where the detected models score poorly on the SecLens benchmark.
4. What is the highest-risk area? Directories with heavy AI authorship in languages where the model has weak coverage.

It is not a vulnerability scanner. It is a mirror.

Quick start

# Scan the current directory
npx ai-authorship scan

# Scan a specific repo
npx ai-authorship scan --path ~/code/my-project

# Limit how many commits to analyze (default 5000)
npx ai-authorship scan --max-commits 1000

# Restrict to one branch
npx ai-authorship scan --branch main

# Machine-readable output
npx ai-authorship scan --json > ai-authorship.json

Share your scan

Generate a 1200×630 PNG card you can post to Twitter, Slack, or drop into a README:

# Default path: <repo>/ai-authorship.png
npx ai-authorship scan --png

# Custom path (file or existing directory)
npx ai-authorship scan --png ./docs/card.png

# Copy the image straight to your clipboard
npx ai-authorship scan --copy

# Hide the repo name (useful for private repos)
npx ai-authorship scan --png --anonymous

The card shows AI authorship percentage, risk grade, top models, and top blind spots. The repo name and contributor data never leave your machine.

Requirements: Node 18+ and a git repository.

What you get

The scan produces a single-screen terminal report:

• AI authorship: overall percentage, with a breakdown of confirmed (commit trailers, tags, co-authors) versus heuristic detections.
• Model distribution: which AI tools and underlying models contributed.
• Repo insights: AI hotspot directories, AI commit size compared to human, contributor mix.
• Blind spots: categories where your detected models underperform on SecLens benchmarks (model × language × OWASP category).
• Code quality: per-language exposure (% AI-authored) and the percentage of vulnerabilities your AI is likely to miss.
• Risk score: an A-through-F grade based on AI coverage and blind-spot severity.
• Tips: concrete prompt and review suggestions tailored to the models in use.

How it works

git log + diffs
   ↓
detectors (trailers, tags, co-authors, message patterns, file/diff heuristics)
   ↓
per-author baselines (calibrate heuristics)
   ↓
SecLens intelligence join (model × category × language)
   ↓
scoring + report

Detection strategies

Six detectors live in src/scanner/detectors/:

| Detector | Signal | Confidence | |---|---|---| | trailer | Generated-By: / Assistant: git trailers | confirmed | | co-author | Co-Authored-By: lines naming Claude, Copilot, Cursor, etc. | confirmed | | tag | [claude], [cursor], [ai] tags in subjects | confirmed | | message-patterns | Conversational or model-style commit prose | heuristic | | conventional-rich | Suspiciously polished conventional commits | heuristic | | files-multiplier | Commits much larger than the author's baseline | heuristic |

A commit is attributed to AI if a confirmed detector fires, or if multiple heuristics agree. Per-author baselines in src/scanner/baselines.ts keep heuristics from over-firing on contributors who normally write large or polished commits.

Intelligence data

Blind-spot data ships in data/seclens-intelligence.json, a snapshot of SecLens benchmark runs (12 models × 8 OWASP categories × 10 languages). Refresh it with:

python3 scripts/extract-seclens.py /path/to/seclens/reports/

Privacy

Everything runs locally. The scan reads git history with git log and never uploads source code, diffs, commit messages, author names, or paths. There is no telemetry. The whole report is generated on your machine from data already in your .git/ directory.

If we ever add an opt-in pattern-sharing feature for the intelligence flywheel, it will be opt-in, anonymized, and called out explicitly in the CLI before the first send.

Development

# Install
npm install

# Run from source
npx tsx src/index.ts scan --path /path/to/repo

# Build single-file bundle
npm run build

# Type check
npm run lint

# Tests
npm test

# Link the local build globally
npm link && ai-authorship scan

Stack

TypeScript (strict, ESM), Node 18+. tsup for bundling, vitest for tests, commander for the CLI, chalk / boxen / cli-table3 for the terminal UI.

Project layout

src/
  cli.ts              # commander setup
  scanner/            # git log parsing, detectors, baselines, insights
  intelligence/       # SecLens data loader + model registry
  scoring/            # risk score + grade
  report/             # terminal renderer
  png/                # share card: sanitize, satori, resvg, PNG
  output/             # flag dispatcher (terminal / json / png / clipboard)
data/
  seclens-intelligence.json
  fonts/              # bundled JetBrains Mono used by the share card
scripts/
  extract-seclens.py        # rebuild intelligence data from SecLens runs
  png-preview.ts            # render every fixture to /tmp for design eyeballing
  png-update-snapshots.ts   # refresh visual-regression baselines

Migrating from @mattersec/vibecheck

Anything you used to run as npx @mattersec/vibecheck ... now runs as npx ai-authorship .... Flags, JSON output, and PNG behavior are unchanged. The only cosmetic difference is the default PNG filename: ai-authorship.png instead of vibe-check.png.

If you pinned the old package in CI, swap the name; if you never did, nothing to do.

Contributing

Contributions welcome, especially new detectors and intelligence improvements.

• Commits: Conventional commits (feat:, fix:, docs:, etc.).
• Style: TypeScript strict, ESM only, no default exports.
• Tests: Add *.test.ts next to the file you change.

License

MIT, see LICENSE.

Built by MatterSec. Questions: [email protected].