check-ai-slop

Find code that need AI/authorship review or quality-risk review.

Not judge. Not proof. Tool splits signals into two buckets: authorship evidence and quality-risk evidence. Human still decide.

What tool do

Scan codebase and recent Git commit metadata. Separate signals:

Authorship evidence

Signals that point toward AI/provenance. Stronger for authorship, still not proof:

• Claude / GPT / Copilot / Cursor attribution text
• was-gpt-here TypeScript isRecord(value: unknown) fingerprint
• AI footer clusters: Generated by AI, Co-authored-by: Claude, AI-assisted
• Git commit markers / actors mentioning Claude, Codex, Cursor, Copilot, Gemini, OpenAI, or Anthropic
• assistant preambles accidentally pasted into code

Quality-risk evidence

Signals that do not prove AI. They catch sloppy/generated-looking risk:

• template leftovers: Replace with your actual ...
• tutorial comments: First, we..., Now, we...
• over-defensive error handling
• weak generated tests and mock-heavy tests
• placeholder secrets and stub code
• missing / hallucinated imports from manifests
• unsafe shortcuts: eval, shell interpolation, TLS off, unsafe YAML, SQL string interpolation, path traversal, regex injection, weak hashes, insecure randomness, Flask debug, JWT bypass, pickle load
• web/auth slop: open redirects, insecure cookies, hardcoded session secrets, raw HTML/template output, permissive CORS
• structure smells: many tiny helpers, low project coupling, duplicate boilerplate, generic names, mutable defaults, lint/type suppressions, cross-language idioms

Use with npx

npx check-ai-slop path/to/repo

JSON:

npx check-ai-slop path/to/repo --json

Fail CI on medium suspicion:

npx check-ai-slop path/to/repo --fail-on medium

Local dev:

npm install
npm run build
node dist/cli.js .

Output mean

Score 0-100. Main score = review priority. Report also shows:

• authorship: evidence that code may carry AI/provenance fingerprints
• quality_risk: security, correctness, maintainability, or test smell evidence

Confidence:

• clean: no useful signal
• low: some smell, inspect
• medium: many signals or strong signal
• high: strong cluster, review before trust

Example:

Review priority: medium (48.0/100)
Authorship evidence 42.0/100, quality-risk evidence 16.0/100.
Scanned 120 files, skipped 3, evidence items 14.

Top flagged files/signals:
- src/foo.ts: medium (48.0/100; authorship=42.0, quality_risk=16.0)
  - Self-admitted AI-generation comment [authorship/self_admitted_ai_comment] (+42.0)
    L3: Generated with Claude

Options

--json                 Print JSON report.
--top <n>              Max flagged files in text mode. Default: 10.
--max-file-bytes <n>   Skip files larger than n bytes. Default: 2097152.
--include-hidden       Include hidden files/directories except vendor/cache ignores.
--no-unknown-imports   Disable dependency-manifest import checks.
--no-git-signals       Disable Git commit metadata provenance checks.
--max-git-commits <n>  Max recent commits to scan for Git signals. Default: 200.
--fail-on <level>      Exit 1 when score reaches low, medium, or high.
--min-score <n>        Exit 1 when score reaches numeric threshold.
--version, -v          Print version.
--help, -h             Print help.

Why not proof

Humans write same bugs and style too. Security tools, tests, detector repos, docs, and policy hooks can contain AI strings on purpose. Git commit metadata can be strong provenance evidence, but it still is not authorship proof. Authorship evidence is stronger than quality-risk evidence, but neither equals proof. Tool report includes false-positive risk for each hit. Read evidence. Decide with context.

Good use

• scan PR before merge
• spot explicit AI commit trailers like Co-authored-by: Claude / Generated with Claude Code
• find generated chunks that need review
• catch unsafe demo-code shortcuts
• catch hallucinated dependency names
• audit test quality when agent made tests

Bad use

• blame person
• auto-reject code with no review
• treat score as authorship proof
• scan detector repo and panic when detector strings match

GitHub Packages

npmjs package name: check-ai-slop.

GitHub Packages requires scoped npm names. Workflow publishes same code as @byigitt/check-ai-slop to npm.pkg.github.com, so package appears under repo Packages tab.

Run GitHub Action: Publish GitHub Package.

Dev

npm test

Tests use Node test runner. Fixtures include real GitHub snippets: one positive AI attribution marker block, one negative clean hand-maintained package snippet, and one unsafe YAML example from Bandit.