AI Code Validator

A lightweight open-source CI plugin that validates AI-generated code for security vulnerabilities, quality issues, and performance problems.

Problem Statement

The Verification Gap Crisis: 96% of developers distrust AI-generated code, yet only 48% verify it before committing. Despite 84% adoption of AI tools, trust is at an all-time low with only 3% of developers "highly trusting" AI output.

AI Code Validator fills the critical gap between AI code generation and human verification by providing automated, contextual validation specifically for AI-generated code patterns.

Features

🔍 AI-Specific Detection

• AI Pattern Detection: Identifies AI-generated code markers and common AI anti-patterns
• AI Vulnerability Scanning: Detects security vulnerabilities commonly introduced by AI tools
• AI Performance Issues: Identifies inefficient code patterns often generated by AI
• AI Confidence Scoring: Measures how likely code is AI-generated

🛡️ Security Validation

• Critical Security Issues: Detects eval(), innerHTML, hardcoded secrets, and other dangerous patterns
• Security Confidence Levels: Severity-based scoring for security concerns
• Contextual Analysis: AI-specific security patterns vs. traditional code smells

📊 Quality Analysis

• Code Quality Scoring: 0-100 quality score based on best practices
• Maintainability Issues: Detects excessive nesting, redundant code, and anti-patterns
• Performance Optimization: Identifies inefficient loops, unnecessary object copying, etc.
• Code Complexity: Analyzes cyclomatic complexity and readability metrics

🚀 CI/CD Integration

• GitHub Actions: Native GitHub Actions workflow support
• GitLab CI: Integrated pipeline support with JUnit reporting
• Jenkins: Compatible with CI/CD pipelines via JSON output
• Multiple Output Formats: JSON, JUnit, GitHub Actions, GitLab CI formats

⚙️ Flexible Configuration

• Custom Rules: Add your own validation rules and patterns
• Threshold Management: Configure quality scores and confidence thresholds
• Selective Enablement: Enable/disable rules based on project needs
• Environment-Aware: Different configurations for development vs. production

Installation

npm install ai-code-validator

Or use it directly via npx:

npx ai-code-validator scan ./src --output report.json --format json

Quick Start

1. Initialize Your Project

# Create default configuration
npx ai-code-validator init

# Initialize with custom config path
npx ai-code-validator init --config ./my-config.json

2. Run Validation

# Scan a directory
npx ai-code-validator scan ./src --output validation-report.json

# Scan with custom threshold
npx ai-code-validator scan ./src --threshold 90 --format github-actions

# Exclude specific files
npx ai-code-validator scan ./src --exclude node_modules --exclude vendor

3. Integrate with CI/CD

GitHub Actions

Add this to your .github/workflows/validate.yml:

name: Code Validation
on: [pull_request]

jobs:
  validate-ai-code:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Validate AI-generated code
        uses: sulthonzh/ai-code-validator@v1
        with:
          path: .
          output: validation-report.json
          format: gitlab-ci
          threshold: 85

GitLab CI

Add this to your .gitlab-ci.yml:

stages:
  - validate

ai-validation:
  stage: validate
  script:
    - npx ai-code-validator scan . --output validation-report.json --format gitlab-ci --threshold 85
  artifacts:
    reports:
      junit: validation-report.xml
    paths:
      - validation-report.json
  allow_failure: false

Configuration

The validator uses a JSON configuration file (default: ai-validator-config.json) with the following structure:

{
  "thresholds": {
    "aiDetectionThreshold": 0.7,
    "minimumQualityScore": 85,
    "failOnCritical": true,
    "failOnHighSeverity": true,
    "maxIssues": 20
  },
  "output": {
    "format": "json",
    "includeDetails": true,
    "quiet": false
  },
  "rules": [
    {
      "id": "ai-insecure-random",
      "name": "AI-generated insecure random number generation",
      "category": "security",
      "severity": "high",
      "enabled": true,
      "pattern": ["Math.random()", "crypto.getRandomValues"],
      "test": "function(code) { return /(Math\\.random\\(\\)|crypto\\.getRandomValues)/.test(code); }",
      "message": "AI often uses Math.random() for security-sensitive operations. Consider crypto.getRandomValues for better randomness.",
      "suggestion": "Use crypto.getRandomValues() for cryptographic operations instead of Math.random()."
    }
  ],
  "ai": {
    "patterns": [
      "/\\s*\\/\\/\\s*AI generated code\\s*\\/",
      "/\\s*\\/\\*\\s*AI generated\\s*\\*\\/\\s*(?:[\\s\\S]*?)\\s*\\/\\*\\s*End AI generated\\s*\\*\\//"
    ],
    "vulnerabilityPatterns": [
      "/eval\\s*\\(",
      "/innerHTML\\s*=/",
      "/document\\.write/"
    ],
    "performancePatterns": [
      "/for\\s*\\(let\\s+i\\s*=\\s*0;\\s*i\\s*<\\s*[^)]+\\.length;\\s*i\\+\\+\\)/i",
      "/JSON\\.parse\\(JSON\\.stringify/"
    ]
  }
}

Configuration Management

# View current configuration
npx ai-code-validator config

# Validate configuration
npx ai-code-validator config --validate

# List all enabled rules
npx ai-code-validator config --list-rules

# Add custom rule
npx ai-code-validator config --add-rule '{"id":"my-rule","name":"My Rule","category":"security","severity":"high","enabled":true,"pattern":["my-pattern"],"test":"function(code){return/code/}","message":"Custom message"}'

# Disable specific rule
npx ai-code-validator config --disable-rule ai-insecure-random

# Enable specific rule
npx ai-code-validator config --enable-rule ai-insecure-random

Output Formats

JSON Format

{
  "summary": {
    "totalFiles": 5,
    "passedFiles": 3,
    "failedFiles": 2,
    "totalIssues": 8,
    "criticalIssues": 2,
    "qualityScore": 78,
    "aiConfidence": 65
  },
  "files": [
    {
      "path": "src/main.js",
      "status": "fail",
      "score": 72,
      "issues": [...],
      "warnings": [...]
    }
  ],
  "violations": [...],
  "recommendations": [
    "🔒 Address security issues immediately",
    "📏 Focus on code quality improvements"
  ]
}

JUnit Format

XML output compatible with CI systems:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="ai-code-validator" tests="5" failures="2" errors="0">
    <testcase name="src/main.js" classname="ai-validator">
      <failure message="AI-generated eval usage" type="security-critical"/>
    </testcase>
  </testsuite>
</testsuites>

GitHub Actions Format

Optimized for GitHub Actions step outputs:

{
  "version": "1.0.0",
  "summary": {
    "status": "failed",
    "passed_files": 3,
    "failed_files": 2,
    "total_issues": 8,
    "critical_issues": 2
  },
  "github": {
    "repository": "owner/repo",
    "ref": "refs/pull/123/merge"
  }
}

Rules and Patterns

Security Rules

• ai-unsafe-eval: Detects AI-generated eval(), Function(), and setTimeout usage
• ai-hardcoded-secrets: Identifies hardcoded credentials, API keys, and secrets
• ai-insecure-random: Detects Math.random() usage in security-sensitive contexts

Quality Rules

• ai-excessive-nesting: Identifies deeply nested code patterns
• ai-redundant-code: Detects unnecessary conditionals and placeholder code
• ai-unnecessary-properties: Identifies inefficient object copying methods

Performance Rules

• ai-inefficient-loop: Detects traditional for loops when array methods would be better
• ai-unnecessary-properties: Identifies inefficient object copying patterns

AI Pattern Detection

• AI Markers: Detects comments and markers indicating AI-generated code
• AI Anti-patterns: Identifies common AI code generation mistakes
• AI Vulnerability Patterns: Scans for security patterns commonly introduced by AI
• AI Performance Patterns: Identifies performance issues frequently generated by AI

Exit Codes

• 0: Success - All files passed validation
• 1: Issues found - Some files failed validation but no critical issues
• 2: Critical issues - Critical security or quality issues detected
• 1: General error - Configuration or execution errors

Development

Building

npm install
npm run build

Testing

npm test
npm run test:watch

Linting

npm run lint
npm run lint:fix

Configuration

# Create development config
cp ai-validator-config.json dev-config.json

# Edit dev-config.json for local development
# Then use: npx ai-code-validator scan ./src --config dev-config.json

Examples

Example: Basic Validation

# Scan your source code
npx ai-code-validator scan ./src --output report.json

# View results
cat report.json | jq '.summary'

Example: CI Integration with GitHub Actions

name: AI Validation
on: [pull_request]

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '18'
      - run: npm install ai-code-validator
      - name: Validate AI code
        run: |
          npx ai-code-validator scan . --output validation-report.json --format github-actions --threshold 85
          cat validation-report.json
      - name: Upload results
        uses: actions/upload-artifact@v3
        with:
          name: validation-results
          path: validation-report.json

Example: Custom Rules

{
  "rules": [
    {
      "id": "custom-no-var",
      "name": "Disallow var declarations",
      "category": "quality",
      "severity": "medium",
      "enabled": true,
      "pattern": ["\\bvar\\s+"],
      "test": "function(code) { return /\\bvar\\s+/.test(code); }",
      "message": "Use let/const instead of var for better scope control",
      "suggestion": "Replace 'var' with 'let' or 'const' based on mutability needs"
    }
  ]
}

Performance

• Fast Scanning: Processes ~1000 lines/second on average
• Memory Efficient: Minimal memory usage with streaming file processing
• Parallel Processing: Multiple files processed concurrently where possible
• Incremental Scanning: Only modified files need re-scanning

Contributing

1. Fork the repository
2. Create a feature branch
3. Add tests for new functionality
4. Run the test suite
5. Submit a pull request

Development Guidelines

• Follow TypeScript best practices
• Add comprehensive tests for new features
• Update documentation for API changes
• Consider performance implications
• Test with various code samples

License

MIT License - see LICENSE for details.

Support

• Issues: Report bugs and request features on GitHub Issues
• Documentation: Full documentation available in the docs/ directory
• Examples: See the examples/ directory for usage patterns

Acknowledgments

• Inspired by the verification gap research from Stack Overflow 2025 survey
• Built to address the growing need for AI-generated code validation
• Community contributions and feedback welcome

AI Code Validator - Bridging the gap between AI generation and human confidence