AutoSpec CLI

AI-powered E2E test generation with Playwright

AutoSpec automatically generates, runs, and fixes Playwright E2E tests using AI. Just point it at your app, and it creates comprehensive test coverage.

Installation

npm install -g autospec-cli

Quick Start

# 1. Initialize in your project
cd my-app
autospec init

# 2. Login to AutoSpec
autospec login

# 3. Start your app
npm run dev  # or your dev command

# 4. Generate tests!
autospec generate

Features

✨ Automatic Test Generation - Scans your codebase and identifies features
🤖 AI-Powered - Uses GPT-4o to write high-quality Playwright tests
🔄 Auto-Fix - Automatically fixes failing tests (up to 5 attempts)
🎯 Behavior-Driven - Extracts user flows and creates realistic test scenarios
📊 Usage Tracking - Built-in limits and analytics

Commands

autospec init

Initialize AutoSpec configuration in your project. Creates .autospec.json:

autospec init

autospec login

Authenticate with AutoSpec API:

autospec login

You'll be prompted for:

• Email
• Password (or create new account)

autospec generate

Generate E2E tests for your application:

autospec generate

Options:

• -f, --feature <name> - Generate test for specific feature
• --max-attempts <number> - Max fix attempts (default: 5)

Example:

autospec generate --feature "User Login" --max-attempts 3

Configuration

AutoSpec creates .autospec.json in your project:

{
  "apiUrl": "https://api.autospec.io",
  "appUrl": "http://localhost:3000",
  "testsDir": "tests/e2e",
  "maxAttempts": 5
}

Options

• apiUrl - AutoSpec API endpoint (default: https://api.autospec.io)
• appUrl - Your local app URL (default: http://localhost:3000)
• testsDir - Where to save tests (default: tests/e2e)
• maxAttempts - Max fix attempts (default: 5)

How It Works

1. Scan - Analyzes your codebase to identify user-facing features
2. Extract - Creates behavior specifications for each feature
3. Generate - Writes Playwright test code
4. Run - Executes tests against your running app
5. Fix - If tests fail, automatically debugs and fixes them

Requirements

• Node.js >= 18
• A running web application (dev server)
• AutoSpec account (free tier available)
• Playwright (automatically set up during autospec init)

Note: Playwright will be automatically installed when you run autospec init. You can choose to install it automatically or follow manual installation instructions. See Playwright documentation for more details.

Example Workflow

# Clone your project
git clone https://github.com/you/your-app
cd your-app

# Install dependencies
npm install

# Install AutoSpec CLI
npm install -g autospec-cli

# Initialize AutoSpec (this will also set up Playwright)
autospec init

# Login
autospec login

# Start your dev server
npm run dev

# Generate tests (in another terminal)
autospec generate

# Tests are created in tests/e2e/
# Run them with:
npx playwright test

Supported Frameworks

AutoSpec works with any web framework:

• ✅ Next.js
• ✅ React
• ✅ Vue
• ✅ Angular
• ✅ Svelte
• ✅ Plain HTML/JS

Troubleshooting

"Not authenticated"

Run autospec login to authenticate.

"Cannot connect to server"

Check that your apiUrl in .autospec.json is correct.

"App not reachable"

Ensure your dev server is running on the appUrl specified in .autospec.json.

Tests still failing after fixes

• Check test output for specific errors
• Ensure selectors are stable
• Verify app is fully loaded before tests run
• Manual adjustments may be needed

Links

• Documentation: github.com/mehdibb/llm-e2e
• Issues: github.com/mehdibb/llm-e2e/issues
• Website: Coming soon!

License

MIT

Happy Testing! 🚀