ai-vet

Catch AI-generated bugs with AST-level precision.

ai-vet is a specialized static analysis tool designed to detect hallucinations, security risks, and logic smells in code generated by LLMs like Copilot, Claude, and ChatGPT. By performing deep AST analysis, it identifies critical issues that traditional linters frequently overlook.

Why ai-vet?

LLMs are powerful but prone to specific failure modes:

• Hallucinating APIs: Inventing methods that look plausible but don't exist.
• Omitting Safety: Forgetting try/catch around network calls or await on async logic.
• Security Slips: Suggesting eval() or hardcoded keys for "simplicity."

ai-vet doesn't just match strings. It uses the TypeScript compiler and Babel to understand the intent of your code, providing a "Senior Reviewer" in your CLI.

Installation

Execute via npx (Recommended):

npx ai-vet audit src/

Global Installation:

npm install -g ai-vet

Project Dependency:

npm install --save-dev ai-vet

Quick Start

# Audit a specific directory
ai-vet audit src/

# Run a project-wide scan
ai-vet scan

# Only scan files changed in your current branch
ai-vet scan --git-diff

# Automatically fix missing-await and innerHTML issues
ai-vet audit src/ --fix

# Explain a specific rule
ai-vet explain hallucinations

Command Reference

| Command | Option | Description | | :----------------- | :------------- | :---------------------------------------------- | | audit [path] | --json | Export results as machine-readable JSON. | | | --quiet | Suppress info messages; show errors only. | | | --fix | Auto-fix deterministic issues. | | scan | --git-diff | Only analyze files changed in git. | | | --json/quiet | Same as audit. | | explain | | View detailed rule logic and code examples. | | init | --yes | Generate a default configuration file. |

Note: --fix currently supports missing-await and security/inner-html rules.

Core Analyzers

🛡️ Phantom Imports & Hallucinations

Detects uninstalled packages and invented exports that don't exist in real libraries.

• TypeScript: Uses ts-morph to resolve actual module exports on disk. If lodash doesn't export magicSort, it's flagged.
• JavaScript: Checks against a curated list of common AI inventions (e.g., useAsyncEffect).

⏳ Missing Await

Identifies unawaited Promises using type-checking.

• Detects assignments without await.
• Identifies "fire-and-forget" calls that should be handled.
• Supports Promise.all and .then()/.catch() chains.

🔒 Security Audit

AST-based detection of dangerous patterns.

• Flags eval() and innerHTML assignments (ignores comments).
• Scans StringLiteral nodes for high-entropy secrets (Stripe, Google Cloud, etc.).

🌐 Error Handling

Specifically targets fetch() logic.

• Ensures every fetch call is wrapped in a TryStatement or has a .catch() block.

🏚️ Deprecated APIs

Precise detection of outdated Node.js traps like url.parse() or new Buffer().

🗑️ Dead Code

Finds unused code that bloats your project.

• Identifies unused imports (named and default) using AST-based reference counting.

🎯 Type Mismatches

Detects obvious type violations in TypeScript.

• Flags primitive assignment errors (e.g., assigning a number to a string).
• Uses the TypeScript compiler to verify assignability with precision.

Configuration

Customize behavior in .aivetrc.json:

{
  "include": ["src/**/*.{js,ts,jsx,tsx,vue}"],
  "exclude": ["node_modules", "dist", "**/*.test.ts"],
  "severity": "HIGH",
  "rules": {
    "hallucinations": "error",
    "missing-await": "error",
    "security": "error"
  }
}

CI/CD & Reporting

ai-vet is built for automation:

• Exit Codes: Returns non-zero for issues at or above your severity threshold.
• Threshold Reporting: Issues below your threshold are visually dimmed in the console, keeping focus on blockers.

Example GitHub Action:

- name: AI Vet Scan
  run: npx ai-vet scan --git-diff

Exit Codes

| Code | Meaning | | :---- | :------------------------------------------- | | 0 | Clean: No issues above threshold detected. | | 1 | Issues found at or above severity threshold. | | 2 | Configuration error (invalid .aivetrc.json). | | 3 | Parse error: AST could not be generated. |

Contributing

Technical contributions are welcome. Please review the project structure and existing test suites before submitting a pull request.

License

MIT License - see LICENSE for details.