Zephyr Scale Automation Agent

Automated agent that fetches test cases from Zephyr Scale with "Automation Candidate" status, generates Playwright test code using AI, and creates a Pull Request.

Features

• 🔍 Fetches test cases from Zephyr Scale with automation candidate status
• 🤖 Generates Playwright test code using AI (OpenAI, Anthropic, Gemini, etc.)
• 📝 Creates test files following your existing codebase patterns
• 🎯 Framework-aware: Automatically analyzes existing test files to match your framework patterns
• 🔀 Creates Git branches and opens Pull Requests automatically
• ⚙️ Configurable via environment variables

Installation

NPM Package

npm install --save-dev @your-org/zephyr-scale-agent

Note: Replace @your-org with your organization's npm scope.

Configuration

1. Create .env file in your project root:

# Zephyr Scale Configuration
ZEPHYR_BASE_URL=https://your-domain.atlassian.net
[email protected]
ZEPHYR_API_TOKEN=your-zephyr-api-token
ZEPHYR_PROJECT_KEY=PROJ
ZEPHYR_AUTOMATION_CANDIDATE_STATUS=Automation Candidate

# AI Configuration
AI_PROVIDER=openai
OPENAI_API_KEY=sk-your-key

# Git Configuration
GIT_BASE_BRANCH=main
GIT_BRANCH_PREFIX=zephyr-auto
GIT_AUTHOR_NAME=Zephyr Automation Agent
[email protected]

# Bitbucket Configuration (optional)
BITBUCKET_ENABLED=true
BITBUCKET_WORKSPACE=your-workspace
BITBUCKET_REPO_SLUG=your-repo
BITBUCKET_USERNAME=your-username
BITBUCKET_APP_PASSWORD=your-app-password

# Output Configuration
TEST_OUTPUT_DIR=tests/api
UI_TEST_OUTPUT_DIR=tests/ui
TEST_FILE_PREFIX=zephyr

2. Copy from example:

   cp node_modules/@your-org/zephyr-scale-agent/env.example.txt .env

Usage

Command Line

# Process all automation candidate test cases
npx @your-org/zephyr-scale-agent

# Process a specific test case
npx @your-org/zephyr-scale-agent --test-case TC-123

# Generate code without creating PR
npx @your-org/zephyr-scale-agent --test-case TC-123 --no-pr

# List all test cases
npx @your-org/zephyr-scale-agent --list-all

# Help
npx @your-org/zephyr-scale-agent --help

NPM Scripts

Add to your package.json:

{
  "scripts": {
    "zephyr:test": "zephyr-agent --test-case",
    "zephyr:all": "zephyr-agent",
    "zephyr:list": "zephyr-agent --list-all"
  }
}

Then use:

npm run zephyr:test TC-123 -- --no-pr
npm run zephyr:all
npm run zephyr:list

Options

• --test-case <key> - Process a specific test case key
• --project <key> - Zephyr Scale project key
• --test-type <type> - Test type: 'api' or 'ui' (default: auto-detect)
• --dry-run - Don't create PR, just generate code
• --no-pr - Skip PR creation (generate code only)
• --list-all - List all test cases and their statuses
• --help - Show help message

How It Works

1. Analyze Framework: Scans existing test files to understand your framework patterns, imports, and structure
2. Fetch Test Cases: Connects to Zephyr Scale API and fetches all test cases with "Automation Candidate" status
3. Extract Test Steps: Retrieves detailed test steps for each test case
4. Generate Code: Uses AI to generate Playwright test code based on test steps and your existing codebase patterns
5. Create Branch: Creates a new Git branch for the test case(s)
6. Commit & Push: Commits the generated test files and pushes to remote
7. Open PR: Creates a Pull Request on Bitbucket (if enabled)

Framework Awareness

The agent automatically analyzes your existing test files to:

• Detect import patterns - Matches your import statements
• Identify test structure - Uses test.describe() vs test(), serial vs parallel
• Find common helpers - Discovers utility functions and base classes
• Match assertions - Uses the same assertion patterns as your existing tests
• Follow setup patterns - Replicates your setup/teardown methods

If no existing tests are found, the agent uses standard Playwright best practices.

Prerequisites

1. Zephyr Scale Access

   • Zephyr Scale API token
   • Project key with test cases

2. AI Provider

   • OpenAI, Anthropic, Gemini, Groq, or Ollama API key

3. Git/Bitbucket

   • Git repository initialized
   • Bitbucket App Password (if using Bitbucket PRs)

Getting API Tokens

Zephyr Scale API Token

1. Go to your Jira instance
2. Navigate to Account Settings → Security → API Tokens
3. Create a new API token
4. Use your email and this token for authentication

Bitbucket App Password

1. Go to https://bitbucket.org/account/settings/app-passwords/
2. Create a new app password
3. Grant "Repositories: Write" permission
4. Copy the generated password

Troubleshooting

"Module not found"

npm install @your-org/zephyr-scale-agent

"Not in a git repository"

Make sure you're running the script from your Playwright project root directory.

"Zephyr Scale API Error"

• Verify your ZEPHYR_BASE_URL is correct
• Check that your API token is valid
• Ensure you have access to the project

"AI generation failed"

• Check your AI provider API key
• Verify AI_PROVIDER is set correctly
• Check API quota/limits

Generated code doesn't match patterns

The AI uses your existing test files as reference. Make sure you have well-structured test files in your test directories for the AI to learn from.

CI/CD Integration

You can run this agent on a schedule using GitHub Actions, Jenkins, or any CI/CD tool. See the deployment guide for examples.

License

ISC

Support

For issues or questions, please check:

• Configuration issues: Review env.example.txt
• API errors: Verify your tokens and permissions
• Code generation: Ensure you have example test files for the AI to learn from