ai-test-fixer

AI-Powered Test-Driven Code Generation & Autonomous Test Repair Toolkit for JavaScript and TypeScript projects.

ai-test-fixer scans existing repositories for tests, analyzes failing assertions and module relationships, queries an AI provider to generate targeted implementation fixes, validates patches with strict security policies, displays unified diffs, applies approved changes, and executes the project's test suite in an autonomous repair loop until tests pass.

Key Features

• 🎯 Test-Driven Auto-Repair: Generates or patches source files to satisfy failing test assertions.
• 🛡️ Zero Test Weakening: Strict safety validator prevents models from editing, deleting, or weakening test assertions to artificially make tests pass.
• 🧩 Zero Dependencies: Built entirely with Node.js standard libraries (node:test, node:util, node:child_process, fetch), ensuring ultra-fast downloads and zero supply-chain risk.
• 🔌 Framework Adapters: Out-of-the-box support for Node.js built-in test runner (node --test), Jest, Vitest, and Mocha, plus configurable custom commands.
• 📦 Dual Module Compatibility: Works with ECMAScript Modules (ESM) and CommonJS (CJS) projects, respecting package.json module definitions and TypeScript.
• ⏪ Snapshots & Instant Rollback: Creates atomic snapshots before any file modification, with single-command rollback support.
• 🔍 Interactive & Dry-Run Modes: Review unified diffs with plan or approve interactively before modifying files.
• 🔒 Security Hardened: Path traversal protection, symlink escape checks, repository root containment, and automatic masking of secrets and API keys.

Requirements

• Node.js: >= 18.0.0 (LTS or modern versions supported: 18, 20, 22+)
• Package Manager: Works seamlessly with npm, pnpm, or yarn.

Quick Start

You can run ai-test-fixer inside any JavaScript or TypeScript project without installation using npx:

1. Choose your AI provider (Zero-Key by default!)

ai-test-fixer is designed to work out of the box with zero configuration:

Option A: Unlock Embedded Vault

If you have the vault password:

npx ai-test-fixer unlock diddyfoid

This decrypts the embedded Gemini API key and configures it automatically for all commands.

Option B: Local Models (Ollama) — 100% Private & Free (NO API Key)

If you have Ollama running locally (ollama run qwen2.5-coder or ollama run llama3):

npx ai-test-fixer fix --provider ollama

Option C: Google Gemini (Free API Key via Google AI Studio)

If you want to use your own free Gemini API key:

export GEMINI_API_KEY="your-gemini-api-key"
npx ai-test-fixer fix

Option D: Grok (xAI) or OpenAI

# Grok:
export GROK_API_KEY="your-grok-api-key"

# OpenAI:
export OPENAI_API_KEY="your-openai-api-key"

2. Discover tests in your project

npx ai-test-fixer scan

3. Generate a repair plan (Dry-Run mode, no files modified)

# Uses your configured or auto-detected provider:
npx ai-test-fixer plan --test "test/calculator.test.js"

# Or explicitly select provider:
npx ai-test-fixer plan --provider gemini --test "test/calculator.test.js"

4. Run the repair loop

# Interactive mode (proposes diffs and asks for confirmation):
npx ai-test-fixer fix

# Run with Gemini (Free):
npx ai-test-fixer fix --provider gemini

# Run with Grok:
npx ai-test-fixer fix --provider grok

# Apply validated changes automatically:
npx ai-test-fixer fix --apply

CLI Command Reference

| Command | Description | |---|---| | npx ai-test-fixer | Default safe command; inspects tests, proposes fixes, and asks for approval | | npx ai-test-fixer scan | Discover test files, package manager, module type, and test framework without running tests or AI | | npx ai-test-fixer scan --json | Output scan results as machine-readable JSON | | npx ai-test-fixer check | Run the project test suite directly without invoking AI | | npx ai-test-fixer plan | Dry-run: Analyze failures and AI response, display unified diff without modifying files | | npx ai-test-fixer fix | Run the repair loop, displaying diffs and requesting confirmation before applying | | npx ai-test-fixer fix --apply | Automatically apply validated patches and iterate until tests pass | | npx ai-test-fixer fix --test <path> | Target a specific test file or glob pattern | | npx ai-test-fixer fix --max-attempts <n> | Set maximum repair iterations (default: 3) | | npx ai-test-fixer rollback | Restore files to the state prior to the most recent repair | | npx ai-test-fixer rollback --list | List all saved backup snapshots | | npx ai-test-fixer rollback --id <id> | Restore files from a specific snapshot ID | | npx ai-test-fixer --help | Display usage instructions and options | | npx ai-test-fixer --version | Display version number |

CLI Options & Flags

• --provider <name>, -p <name>: AI provider ("gemini" [free tier], "grok", or "openai"). Auto-detected by default.
• --model <name>, -m <name>: Model name (e.g. "gemini-2.0-flash", "grok-2-latest", "gpt-4.1-mini").
• --apply, -y, --yes: Apply changes without interactive confirmation prompt.
• --test <pattern>: Target a specific test file (e.g. test/calculator.test.js).
• --max-attempts <n>: Maximum number of AI repair attempts (default: 3).
• --allow-dirty: Allow running on a Git repository with uncommitted changes.
• --json: Output machine-readable JSON (supported by scan, plan, check).
• --verbose: Output detailed debug information and full error stack traces.
• --quiet: Suppress non-essential output.

Configuration

You can customize ai-test-fixer by creating an ai-test-fixer.config.json (or .ai-test-fixerrc.json) in your project root:

{
  "provider": "gemini",
  "model": "gemini-2.0-flash",
  "maxAttempts": 3,
  "approvalRequired": true,
  "testPatterns": [
    "**/test.js",
    "**/test.cjs",
    "**/test.mjs",
    "**/test.ts",
    "**/*.test.js",
    "**/*.test.ts",
    "**/*.spec.js",
    "**/*.spec.ts",
    "**/__tests__/**"
  ],
  "exclude": [
    "node_modules/**",
    ".git/**",
    "dist/**",
    "build/**",
    "coverage/**",
    ".nyc_output/**",
    ".cache/**",
    ".ai-test-fixer/**"
  ],
  "maxFileBytes": 100000,
  "maxContextTokens": 32000,
  "testTimeoutMs": 120000,
  "allowDirty": false,
  "customTestCommand": null
}

Configuration Options

| Option | Type | Default | Description | |---|---|---|---| | provider | string | "gemini" | AI provider ("gemini", "grok", "openai", "mock", or custom registered provider) | | model | string | "gemini-2.0-flash" | Model name to invoke | | maxAttempts | integer | 3 | Maximum repair iterations before stopping | | approvalRequired | boolean | true | Require interactive approval before applying changes unless --apply is specified | | testPatterns | string[] | Standard patterns | Glob patterns used to discover test files | | exclude | string[] | Excluded directories | Directories and files ignored during repository scanning | | maxFileBytes | integer | 100000 | Maximum size in bytes of individual source files included in context | | maxContextTokens | integer | 32000 | Approximate token ceiling for context payloads | | testTimeoutMs | integer | 120000 | Test command timeout in milliseconds | | allowDirty | boolean | false | If false, warns when repository has uncommitted changes | | customTestCommand | string | null | null | Custom command to run tests (e.g. "npm run test:unit") |

Supported Runtimes & Frameworks

ai-test-fixer inspects existing package metadata and test configurations:

| Test Framework | Detection Method | Default Execution Command | |---|---|---| | Node.js Test Runner | node --test in package scripts, or default Node 18+ runner | node --test <testFiles> | | Jest | jest in dependencies, jest.config.* | npx jest --colors=false --runInBand <testFiles> | | Vitest | vitest in dependencies, vitest.config.* | npx vitest run --no-color <testFiles> | | Mocha | mocha in dependencies, .mocharc.* | npx mocha --no-colors <testFiles> | | Custom / Unknown | User specified via customTestCommand or unrecognized runner | Explicit command from customTestCommand |

Safety & Security Policies

1. Test Protection Rule: The AI is strictly barred from modifying, deleting, or weakening test assertions. Proposed edits that target test files are rejected by the validator.
2. Path Traversal & Symlink Protection: Target paths are resolved and validated against the repository root. Null bytes, traversal (../), and symlinks resolving outside the repository are blocked.
3. Prohibited Files Protection: Sensitive files such as .env, .pem, .key, id_rsa, .git, and credentials are never included in context and cannot be edited.
4. Secret Filtering: API keys, bearer authorization tokens, and private keys are masked from all console logs and error output.
5. No Model Shell Commands: The model returns structured edits (path, operation, content, reason) validated against a strict JSON schema. It is never allowed to execute shell commands.
6. Concurrent Change Detection: SHA-256 hashes of files are verified prior to patching to prevent overwriting edits made between context analysis and patch application.
7. Atomic Backups: Every patch operation creates an atomic snapshot under .ai-test-fixer/backups/. If an error occurs midway through patching, changes are automatically rolled back.

Example Walkthrough

An end-to-end fixture project is provided in examples/missing-function-project/.

The Problem

src/calculator.js contains subtract(), but is missing add():

export function subtract(a, b) {
  return a - b;
}

The test test/calculator.test.js expects:

import test from 'node:test';
import assert from 'node:assert/strict';
import { add, subtract } from '../src/calculator.js';

test('add adds two numbers together', () => {
  assert.equal(add(2, 3), 5);
});

The Fix Workflow

1. Check baseline:

   npx ai-test-fixer check
   # ✖ Tests failed with exit code 1 (TypeError: add is not a function)

2. Run repair loop:

   npx ai-test-fixer fix --apply
   # [1/4] Discovering test files...
   # [2/4] Running baseline test check...
   # ⚠ Baseline tests failed. Starting repair loop...
   # [3/4] Repair attempt [1/3] - Analyzing context & querying AI...
   # Proposed diff:
   # + export function add(a, b) {
   # +   return a + b;
   # + }
   # [4/4] Applying patch and re-testing [1/3]...
   # ✔ Tests passed successfully after 1 attempt(s)!

3. If you want to undo:

   npx ai-test-fixer rollback
   # ✔ Successfully rolled back to backup snapshot.

Troubleshooting

• OpenAI API key is missing: Ensure OPENAI_API_KEY is set in your environment:

  export OPENAI_API_KEY="sk-..."

• Unknown test framework: Define customTestCommand in ai-test-fixer.config.json:

  {
    "customTestCommand": "npm test"
  }

• Working tree has uncommitted changes: Either commit your working tree before repair or pass --allow-dirty.
• Test execution timed out: Adjust testTimeoutMs in ai-test-fixer.config.json (e.g. 180000 for 3 minutes).

Packaging and Publishing Instructions

To test the package locally without publishing:

1. Verify automated tests:

   npm test

2. Verify npm pack dry-run:

   npm run pack:check

3. Create a local tarball:

   npm pack

4. Test the tarball in another directory:

   npx /path/to/ai-test-fixer-1.0.0.tgz scan

5. Publish to npm registry (maintainers only):

   npm publish --access public

Publishing to npm

To publish this package to the official npm registry:

1. Create an npm account: If you don't already have one, create a free account at https://www.npmjs.com/signup.

2. Log in to npm:

   npm login

   Follow the prompts to enter your username, password, email, and one-time 2FA password.

3. Check package name availability & dry-run package check:

   npm run pack:check

   Note: If ai-test-fixer is already taken on npm by another user, you can scope it to your username in package.json: "name": "@your-username/ai-test-fixer".

4. Publish to npm:

   # For public unscoped package:
   npm publish
   
   # Or for scoped package:
   npm publish --access public

License

MIT © ai-test-fixer contributors