Fix Tests AI 🔧

Automatically fix failing tests using Claude AI. No more debugging test failures manually!

Features

• 🔍 Auto-detects test runner (Jest, Vitest, Mocha, AVA, Tape)
• 🤖 AI-powered analysis using the latest Claude 4.5 Sonnet
• 🔧 Automatic fixing applies fixes to your code
• 🔄 Verification loop re-runs tests to confirm
• 🎯 One test at a time focused and narrow
• 🌈 Dry-run mode preview changes first
• 🛠️ Diagnostic mode verify your setup with --check
• 📁 Recursive env loading finds your .env.local automatically

Quick Start

1. Install globally

npm install -g fix-tests-ai

2. Set up API key

The easiest way is to create a .env.local file in your project:

echo 'ANTHROPIC_API_KEY="sk-ant-..."' >> .env.local

Or secure it globally in your shell config:

echo 'export ANTHROPIC_API_KEY="sk-ant-..."' >> ~/.zshrc # or ~/.bashrc

[!TIP] Use fix-tests --check to verify your API key and connection.

3. Run in any project with failing tests

cd your-project
fix-tests

That's it! 🎉

Usage

Basic Usage

# Fix failing tests automatically
fix-tests

# Interactive mode - review each fix before applying (recommended)
fix-tests --interactive

# Preview changes without applying
fix-tests --dry-run

# Run diagnostics to verify your setup and API key
fix-tests --check

# Show help
fix-tests --help

# Setup instructions
fix-tests --setup

Example Session

$ npm test
FAIL tests/calculator.test.ts
  ● should add two numbers
    Expected: 4, Received: 5

$ fix-tests

╔════════════════════════════════════════════════════════════╗
║              🔧 Fix Tests AI - by Claude                  ║
╚════════════════════════════════════════════════════════════╝

🔍 Detecting test runner...
✅ Found: jest
   Command: npm test

🧪 Running tests: npm test

📊 Analyzing test failures...
✅ Found 1 failing test(s)

Failed tests:
  1. should add two numbers
     tests/calculator.test.ts

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
🔧 Fixing: should add two numbers
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

🔍 Analyzing: should add two numbers
📄 Test file: tests/calculator.test.ts
📄 Source file: src/calculator.ts

🤖 Asking Claude to analyze the failure...

💡 Claude's Analysis:
The add function has an off-by-one error. Line 5 is adding 
an extra 1: return a + b + 1; should be: return a + b;

📝 Fixing: src/calculator.ts
✅ Applied fix to src/calculator.ts

🔄 Re-running tests to verify fix...

═══════════════════════════════════════════════════════════════
✅ SUCCESS! All tests now passing!
═══════════════════════════════════════════════════════════════

Interactive Mode 🎯

Interactive mode gives you full control over which fixes to apply. Review Claude's analysis, see a colored diff, and approve or reject each fix before it's applied.

Usage

fix-tests --interactive
# or
fix-tests -i

Interactive Workflow

When a fix is generated, you'll see:

• 🤖 Claude's Analysis - Explanation of the bug and fix
• 📊 Confidence Level - High/Medium/Low confidence rating
• 📝 Colored Diff - Preview of changes (red = removed, green = added)

Then choose:

• [A]pply - Apply this fix and continue
• [R]eject - Skip this fix and exit
• [S]kip - Skip to next failure (if multiple)
• [V]iew - View full diff with context
• [Q]uit - Exit without applying

Example Interactive Session

$ fix-tests --interactive

🔧 Fixing: should add two numbers
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

🤖 Claude's Analysis:
The add function has an off-by-one error. It's adding an extra 1.

Confidence: high
File to fix: calculator.js

📝 Proposed Changes:
────────────────────────────────────────────────────────────
-    6 | return a + b + 1;
+    6 | return a + b;
────────────────────────────────────────────────────────────

What would you like to do?
  [A]pply this fix
  [R]eject this fix
  [S]kip to next failure
  [V]iew full diff
  [Q]uit

Your choice: a

✓ Applying fix...
✅ Applied fix to calculator.js

🔄 Re-running tests to verify fix...

═══════════════════════════════════════════════════════════════
✅ SUCCESS! All tests now passing!
═══════════════════════════════════════════════════════════════

How It Works

1. Detects your test runner from package.json
2. Runs your tests and captures failures
3. Parses the test output (runner-specific)
4. Analyzes with Claude AI (test + source code)
5. Applies the suggested fix
6. Verifies by re-running tests
7. Retries if needed (up to 3 attempts)

Supported Test Runners

| Test Runner | Status | Auto-detected | |------------|--------|---------------| | Jest | ✅ | ✅ | | Vitest | ✅ | ✅ | | Mocha | ✅ | ✅ | | AVA | ✅ | ✅ | | Tape | ✅ | ✅ | | Others | ⚠️ Generic | - |

API Key Setup

Option 1: .env.local (Recommended for Local Dev)

The tool recursively searches for .env and .env.local files in parent directories. This is the primary way to manage variables locally.

[!NOTE] .env.local is included in .gitignore by default to ensure your API keys are never committed to your repository.

# In your project root
echo 'ANTHROPIC_API_KEY=sk-ant-...' >> .env.local

Option 2: Environment Variable

export ANTHROPIC_API_KEY="sk-ant-..."

Option 3: Shell Config (Persistent)

# Bash
echo 'export ANTHROPIC_API_KEY="sk-ant-..."' >> ~/.bashrc
source ~/.bashrc

# Zsh
echo 'export ANTHROPIC_API_KEY="sk-ant-..."' >> ~/.zshrc
source ~/.zshrc

Troubleshooting

"No API key found"

Solution: Set the ANTHROPIC_API_KEY environment variable

export ANTHROPIC_API_KEY="sk-ant-..."
fix-tests

"Could not detect test runner"

Solution: The tool will fall back to npm test. Make sure you have a test script in package.json:

{
  "scripts": {
    "test": "jest"
  }
}

"Tests failed but could not parse output"

Solution: Your test runner format might not be supported. Open an issue with the output format.

"Claude API error"

Solutions:

• Check your API key is valid
• Ensure you have API credits
• Check network connection

Limitations (v0.1.0)

• ❌ Fixes one test at a time (run multiple times for multiple failures)
• ❌ Works best with unit tests (not complex E2E tests)
• ❌ May not fix tests requiring external service setup
• ❌ Requires valid Anthropic API key (costs ~$0.01-0.10 per fix)

Roadmap

• [x] Interactive mode (approve each fix) ✨ NEW in v0.1.0
• [ ] Fix multiple tests in one run
• [ ] Better diff visualization
• [ ] Support more test runners
• [ ] Rollback on failed fix
• [ ] Configuration file support
• [ ] CI/CD integration guide

Development

Clone & Build

git clone https://github.com/starslingdev/refactored-palm-tree
cd fix-tests-ai
npm install
npm run build

Test Locally

We've provided a demo script to quickly test the tool in a sandbox:

# 1. Setup the demo project
./demo-setup.sh

# 2. Enter the demo
cd fix-tests-demo

# 3. Fix the tests
node ../dist/cli.js

Mimic Real NPM Installation

To test the package as if it were installed via NPM:

Method 1: Using npm link (Recommended)

This creates a symlink so your changes are reflected immediately.

# In the fix-tests-ai directory
npm link

# In your project directory
npm link fix-tests-ai
fix-tests

Method 2: Global Installation from Source

This mimics a true global installation.

# In the fix-tests-ai directory
npm install -g .

You can now run fix-tests from anywhere on your system.

Project Structure

fix-tests-ai/
├── src/
│   ├── cli.ts         # CLI entry point
│   ├── detector.ts    # Test runner detection
│   ├── runner.ts      # Run tests
│   ├── parser.ts      # Parse test output
│   ├── fixer.ts       # Claude API + apply fixes
│   ├── types.ts       # TypeScript types
│   └── index.ts       # Exported API
├── package.json
├── tsconfig.json
└── README.md

Contributing

Contributions welcome! Please:

1. Fork the repo
2. Create a feature branch
3. Add tests (when we have them 😅)
4. Submit a PR

Cost

Claude API pricing (as of 2024):

• Sonnet 4: ~$0.003 per 1K input tokens, ~$0.015 per 1K output tokens
• Estimated cost per fix: $0.01 - $0.10 depending on code size

Much cheaper than your time debugging! ⏰💰

License

MIT

Credits

Built with:

• Claude AI by Anthropic
• TypeScript
• Node.js