@zibby/cli

Command-line interface for Zibby Test Automation with Rails-like generators.

Installation

# Global installation
npm install -g @zibby/cli

# Or use npx
npx @zibby/cli init my-project

Authentication

Zibby CLI requires two-layer authentication for cloud uploads:

Setup Options

Option 1: Local Development (Interactive)

# Step 1: Login with OAuth
zibby login
# Opens browser → Login → Saves to ~/.zibby/config.json

# Step 2: Set project token in .env
# ZIBBY_API_KEY=zby_xxx

Option 2: CI/CD (Personal Access Token)

# Step 1: Generate token at https://zibby.app/settings/tokens

# Step 2: Set environment variables
export ZIBBY_USER_TOKEN=zby_pat_xxxxx
export ZIBBY_PROJECT_ID=zby_xxxxx

# Step 3: Run tests
zibby run tests/**/*.spec.js

Running Tests with Cloud Sync

# Both project token + user token are required
zibby run test.spec.js --sync

# Or with explicit project override
zibby run test.spec.js --project zby_xxx --sync

Backend validates:

• ✅ Project token is valid (zby_xxx)
• ✅ User token is valid (JWT from zibby login OR PAT from env var)
• ✅ User is a member of the project

Authentication Commands

# Interactive login (local development)
zibby login

# Check current status
zibby status

# Logout (clears session)
zibby logout

Personal Access Tokens (CI/CD)

For automated pipelines, use Personal Access Tokens:

1. Generate token: https://zibby.app/settings/tokens
2. Add to CI secrets:

   # GitHub Actions
   env:
     ZIBBY_USER_TOKEN: ${{ secrets.ZIBBY_USER_TOKEN }}
     ZIBBY_PROJECT_ID: zby_xxxxx

3. Run tests:

   npm install -g @zibby/cli
   zibby run tests/**/*.spec.js

Token features:

• 📅 Configurable expiration (30/60/90 days, or never)
• 🔄 Revoke anytime from dashboard
• 📊 Track last used timestamp
• 🔐 Secure: hashed storage, shown once

Benefits of Two-Layer Auth

| Benefit | Description | |---------|-------------| | 🔒 Enhanced Security | Both project token AND user token required | | 📊 Audit Trail | Tracks WHO uploaded each test execution | | 👥 Team Access Control | Revoke users without rotating project token | | 🚫 Project Membership | Can't upload to projects you're not a member of | | 🤖 CI/CD Ready | Personal Access Tokens for automation |

Commands

zibby init - Project Generator

Initialize a new test automation project (like rails new):

# Create new project
zibby init my-tests

# Or initialize in current directory
zibby init

# Options
zibby init my-tests --agent=cursor --no-cloud --skip-install

What it does:

• ✅ Creates project structure
• ✅ Generates .zibby.config.js
• ✅ Generates .env.example
• ✅ Creates example test specs
• ✅ Installs dependencies
• ✅ Installs Playwright browsers
• ✅ Generates README

zibby run - Run Test Specification

Execute a test specification:

# Basic usage
zibby run test-specs/auth/login.txt

# With options
zibby run test-specs/auth/login.txt --agent=cursor --headless

# Open results in browser after completion
zibby run test-specs/auth/login.txt --project zby_xxx --collection "My Tests" --open

# For CI/CD
zibby run test-specs/auth/login.txt --agent=cursor --auto-approve --headless

Options:

• --agent <type> - Agent to use (claude, cursor)
• --headless - Run browser in headless mode
• --config <path> - Path to config file (default: .zibby.config.js)
• --auto-approve - Auto-approve MCP tools for CI/CD
• -o, --open - Open test results in browser after upload completes
• --project <id> - Project API key for cloud sync (or use ZIBBY_API_KEY env)
• --collection <name> - Collection to upload to
• --sync / --no-sync - Force enable/disable cloud sync

zibby video - Organize Test Videos

Move test videos next to their test files:

zibby video

Before:

test-results/auth-login-Login-Functionality-chromium/video.webm

After:

tests/auth/login.spec.webm

zibby ci-setup - CI/CD Setup

Setup Cursor Agent for CI/CD:

# Patch cursor-agent for auto-approval
zibby ci-setup

# Get approval keys
zibby ci-setup --get-keys

# Save approval keys to project
zibby ci-setup --get-keys --save

Generated Project Structure

When you run zibby init, it creates:

my-project/
├── .zibby.config.js          # Configuration (50 lines)
├── .env.example               # API key templates
├── .gitignore                 # Proper ignores
├── package.json               # With zibby dependencies
├── playwright.config.js       # Auto-configured
├── README.md                  # Full instructions
├── test-specs/                # Test specifications
│   └── examples/
│       └── google-search.txt
├── tests/                     # Generated tests (created on first run)
└── test-results/              # Test artifacts

Configuration

The .zibby.config.js file:

export default {
  agent: {
    provider: 'claude',  // or 'cursor'
    
    claude: {
      model: 'claude-sonnet-4-20250514',
      maxTokens: 4096,
    },
  },

  paths: {
    specs: 'test-specs',
    generated: 'tests',
  },

  context: {
    discovery: {
      global: 'CONTEXT.md',
      pathBased: 'test-specs/{segment}/CONTEXT.md',
      env: `env-${process.env.ENV || 'local'}.js`,
    }
  },

  // Video recording (affects playwright.config.js generation)
  video: 'on',  // Options: 'off', 'on', 'retain-on-failure', 'on-first-retry'

  // Browser viewport size for video recording and testing
  // Default: 1280x720 (works well on most screens)
  // For larger displays: { width: 1920, height: 1080 }
  // For mobile testing: { width: 375, height: 667 } (iPhone SE)
  viewport: { width: 1280, height: 720 },

  // Playwright artifacts (screenshots, traces, videos)
  // Set to false to disable, or a path like 'test-results/playwright-artifacts' to enable
  playwrightArtifacts: false,

  // Cloud sync - auto-upload test results & videos
  cloudSync: false
};

Examples

Generate Project and Run Test

# Generate project
zibby init my-tests
cd my-tests

# Add API key
cp .env.example .env
# Edit .env: ANTHROPIC_API_KEY=sk-ant-...

# Run example test
zibby run test-specs/examples/google-search.txt

# Run with cloud sync and open results
zibby run test-specs/examples/google-search.txt \
  --project zby_xxx \
  --collection "My Tests" \
  --open

# Organize videos
zibby video

# Run generated test
npx playwright test tests/examples/google-search.spec.js

CI/CD Usage

With Claude Agent (Simple)

# Just set API key in CI environment
export ANTHROPIC_API_KEY=sk-ant-...

# Run tests
zibby run test-specs/auth/login.txt --agent=claude --headless

With Cursor Agent (Requires Setup)

# 1. Install cursor-agent (in CI container/runner)
curl https://cursor.com/install -fsS | bash

# 2. Set Cursor API key
export CURSOR_API_KEY=your-cursor-token-here

# 3. Patch cursor-agent for auto-approval (one-time setup)
zibby ci-setup

# 4. Get and save approval keys (optional, for faster subsequent runs)
zibby ci-setup --get-keys --save

# 5. Run tests
zibby run test-specs/auth/login.txt \
  --agent=cursor \
  --auto-approve \
  --headless

# 6. Upload results to cloud (optional)
zibby run test-specs/auth/login.txt \
  --agent=cursor \
  --auto-approve \
  --headless \
  --sync \
  --collection="CI Tests"

GitHub Actions Example:

name: Test with Cursor Agent

on: [push]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: '18'
      
      - name: Install dependencies
        run: npm install
      
      - name: Install cursor-agent
        run: curl https://cursor.com/install -fsS | bash
      
      - name: Setup Cursor Agent for CI
        run: npx zibby ci-setup
        env:
          CURSOR_API_KEY: ${{ secrets.CURSOR_API_KEY }}
      
      - name: Run tests
        run: |
          npx zibby run test-specs/auth/login.txt \
            --agent=cursor \
            --auto-approve \
            --headless \
            --sync
        env:
          CURSOR_API_KEY: ${{ secrets.CURSOR_API_KEY }}
          ZIBBY_API_KEY: ${{ secrets.ZIBBY_API_KEY }}

Using the Cursor Agent

The Cursor Agent integrates with the cursor-agent CLI to leverage Cursor's AI capabilities for test generation. This is ideal for teams already using Cursor IDE.

Setup for Cursor Agent

Local Development (with Cursor IDE)

If you have Cursor IDE installed, no additional setup is needed! The CLI will use your stored Cursor credentials automatically.

# 1. Initialize project with cursor agent
zibby init my-tests --agent=cursor

# 2. Run tests
cd my-tests
zibby run test-specs/examples/google-search.txt --agent=cursor

CI/CD Setup

For CI/CD environments, you need to:

1. Install cursor-agent CLI:

curl https://cursor.com/install -fsS | bash
# or
npm install -g cursor-agent

2. Get your Cursor API token:

   • Visit https://cursor.com/settings
   • Copy your API token

3. Set environment variable:

export CURSOR_API_KEY=your-cursor-token-here

4. Patch cursor-agent for auto-approval:

zibby ci-setup

This patches the cursor-agent binary to automatically approve MCP tool calls, which is required for automated pipelines.

Cursor Agent Examples

Basic Test Generation

# Generate test using Cursor Agent
zibby run test-specs/auth/login.txt --agent=cursor

# With options and auto-open results
zibby run test-specs/auth/login.txt \
  --agent=cursor \
  --headless \
  --collection="Auth Tests" \
  --open

# Upload to specific project
zibby run test-specs/auth/login.txt \
  --agent=cursor \
  --project zby_your_project_id \
  --collection="My Tests" \
  -o

Running in CI/CD

# Complete CI/CD workflow
zibby run test-specs/auth/login.txt \
  --agent=cursor \
  --auto-approve \
  --headless \
  --sync

Running Specific Workflow Nodes

# Run only live execution (useful for debugging)
zibby run test-specs/auth/login.txt \
  --agent=cursor \
  --node=execute_live

# Resume from last session and run script generation
zibby run test-specs/auth/login.txt \
  --agent=cursor \
  --node=generate_script \
  --session=last

Using Custom Workflows

# Download workflow from cloud
zibby workflow download --type=run_test --include-default

# Modify .zibby/workflow-run_test.json as needed

# Run with custom workflow
zibby run test-specs/auth/login.txt \
  --agent=cursor \
  --workflow=CustomWorkflow

Cursor Agent vs Claude Agent

| Feature | Cursor Agent | Claude Agent | |---------|--------------|--------------| | Setup | Requires cursor-agent CLI | Just API key | | Local Dev | Uses IDE credentials | Requires API key | | CI/CD | Requires patching + token | Just API key | | Speed | Depends on IDE connection | Direct API (faster) | | Cost | Uses Cursor subscription | Pay per token | | Best For | Teams using Cursor IDE | Pure CLI workflows |

Approval Keys

Cursor Agent uses MCP tool approval keys to determine which tools can be auto-approved. You can manage these keys:

# Get current approval keys
zibby ci-setup --get-keys

# Save to project
zibby ci-setup --get-keys --save

# Keys are saved to:
# - .mcp/approval-keys.json (project-specific)
# - ~/.cursor-agent/approval-keys.json (global fallback)

Troubleshooting Cursor Agent

"cursor-agent not found"

# Install cursor-agent
curl https://cursor.com/install -fsS | bash

# Verify installation
cursor-agent --version

"CURSOR_API_KEY not set" (CI/CD only)

# Get token from https://cursor.com/settings
export CURSOR_API_KEY=your-token-here

"Tool approval required" (CI/CD)

# Patch cursor-agent for auto-approval
zibby ci-setup

# Verify patch
zibby ci-setup --get-keys

Tests hang or timeout

# Run in headed mode to see what's happening
zibby run test-specs/auth/login.txt --agent=cursor

# Check session logs
ls -la test-results/sessions/

MCP connection errors

# Ensure Playwright MCP is set up
zibby setup-playwright --headed

# Check MCP config
cat .mcp/config.json

Features

✅ Interactive Prompts - Choose agent, MCP, cloud sync ✅ Zero Configuration - Works out of the box ✅ Beautiful CLI - Colorful output with progress indicators ✅ CI/CD Ready - Auto-approval for automated pipelines ✅ Video Organization - Keep videos next to tests ✅ Multi-Agent Support - Claude (API) or Cursor (CLI)

Environment Variables

Agent Configuration

# ============================================
# Claude Agent (Anthropic API)
# ============================================
ANTHROPIC_API_KEY=sk-ant-...
# Required: Always
# Get from: https://console.anthropic.com/

# ============================================
# Cursor Agent (cursor-agent CLI)
# ============================================
CURSOR_API_KEY=your-cursor-token-here
# Required: Only in CI/CD (not needed locally with Cursor IDE)
# Get from: https://cursor.com/settings
# Install cursor-agent: curl https://cursor.com/install -fsS | bash

# ============================================
# Optional: Cloud Sync
# ============================================
ZIBBY_API_KEY=zby_live_...
# Optional: For uploading test results and videos to Zibby Cloud
# Get from: https://app.zibby.com/projects (copy project API key)

ZIBBY_PROJECT_ID=proj_...
# Optional: Specific project ID (auto-detected from API key if not set)

ZIBBY_TENANT_ID=org_...
# Optional: Organization/tenant ID

Complete .env.example

# Copy this to .env and fill in your keys

# ===========================================
# AI AGENT (choose one)
# ===========================================

# Option 1: Claude Agent (recommended for CI/CD)
ANTHROPIC_API_KEY=sk-ant-your-key-here

# Option 2: Cursor Agent (for teams using Cursor IDE)
# Local: Not needed (uses IDE credentials)
# CI/CD: Required
# CURSOR_API_KEY=your-cursor-token-here

# ===========================================
# CLOUD SYNC (optional)
# ===========================================

# Upload test results to Zibby Cloud
# ZIBBY_API_KEY=zby_live_your-project-key
# ZIBBY_PROJECT_ID=proj_your-project-id

# ===========================================
# ENVIRONMENT (optional)
# ===========================================

# For environment-specific configs
# NODE_ENV=development|production|staging

Advanced Cursor Agent Usage

Running Multiple Tests

# Run all tests in a folder
for spec in test-specs/auth/*.txt; do
  zibby run "$spec" --agent=cursor --headless --sync
done

# Or use a workflow system
zibby workflow download --type=run_test
# Edit workflow to run multiple specs
zibby run test-specs/auth/login.txt --workflow=BatchRunner

Custom Approval Keys

By default, cursor-agent requires approval for each MCP tool call. For CI/CD, you can configure which tools are auto-approved:

# 1. Get current keys
zibby ci-setup --get-keys

# 2. Edit approval-keys.json
cat > .mcp/approval-keys.json << 'EOF'
{
  "playwright": {
    "browser_navigate": true,
    "browser_click": true,
    "browser_type": true,
    "browser_screenshot": true,
    "browser_close": true
  }
}
EOF

# 3. Use in CI
zibby run test-specs/auth/login.txt --agent=cursor --auto-approve

Session Management

Cursor Agent creates session folders for each run. You can reuse sessions for debugging:

# 1. Run test (creates session folder)
zibby run test-specs/auth/login.txt --agent=cursor

# Session saved to: test-results/sessions/1709567890/

# 2. View session files
ls test-results/sessions/1709567890/execute_live/
# events.json - All MCP tool calls and responses
# videos/ - Recorded video(s)
# title.txt - Generated test name

# 3. Resume from specific session
zibby run test-specs/auth/login.txt \
  --agent=cursor \
  --node=generate_script \
  --session=1709567890

# 4. Or use "last" to resume from most recent
zibby run test-specs/auth/login.txt \
  --agent=cursor \
  --node=generate_script \
  --session=last

Debugging Cursor Agent

# 1. Run in headed mode (see browser)
zibby run test-specs/auth/login.txt --agent=cursor

# 2. Check cursor-agent logs
tail -f ~/.cursor-agent/logs/agent.log

# 3. Inspect MCP tool calls
cat test-results/sessions/*/execute_live/events.json | jq '.[] | select(.type == "tool_call")'

# 4. Verify MCP setup
cat .mcp/config.json

# 5. Test MCP connection
cursor-agent --test-mcp

Performance Optimization

# Use headless mode (faster)
zibby run test-specs/auth/login.txt --agent=cursor --headless

# Skip video recording
# Edit .zibby.config.js:
# video: 'off'

# Run specific nodes only
zibby run test-specs/auth/login.txt \
  --agent=cursor \
  --node=execute_live  # Skip script generation & verification

# Reuse existing session
zibby run test-specs/auth/login.txt \
  --agent=cursor \
  --node=generate_script \
  --session=last  # No need to execute live again

Best Practices for Cursor Agent

1. Local Development: Use Cursor IDE credentials (no CURSOR_API_KEY needed)
2. CI/CD: Always set CURSOR_API_KEY and patch cursor-agent
3. Auto-Approval: Use --auto-approve flag in CI/CD only
4. Headless Mode: Always use --headless in CI/CD for performance
5. Session Reuse: Use --session=last to debug without re-running
6. Cloud Sync: Use --sync to automatically upload results
7. Collections: Use --collection to organize tests by feature/module

Comparing Agents

When to use Cursor Agent:

• ✅ Your team uses Cursor IDE
• ✅ You want to leverage Cursor's AI capabilities
• ✅ You're okay with CLI-based execution
• ✅ You have a Cursor subscription

When to use Claude Agent:

• ✅ You want the simplest setup
• ✅ You prefer direct API access
• ✅ You need faster execution
• ✅ You're running in pure CI/CD (no IDE)

Recommendation: Start with Claude Agent for simplicity. Switch to Cursor Agent if your team already uses Cursor IDE and wants to leverage its AI capabilities.

Opening Test Results

The --open (or -o) flag automatically opens test results in your browser after upload completes:

# Opens browser to test results page
zibby run test-specs/auth/login.txt \
  --project zby_xxx \
  --collection "My Tests" \
  --open

# Short form
zibby run test-specs/auth/login.txt --project zby_xxx -o

Environment-aware URLs:

• Local: http://localhost:3000/projects/{projectId}/runs/{executionId}
• Staging: https://app-staging.zibby.app/projects/{projectId}/runs/{executionId}
• Production: https://app.zibby.app/projects/{projectId}/runs/{executionId}

Environment configuration:

# Default: local
ZIBBY_ENV=local

# Staging
ZIBBY_ENV=staging
ZIBBY_STAGING_FRONTEND_URL=https://app-staging.zibby.app

# Production
ZIBBY_ENV=prod
ZIBBY_PROD_FRONTEND_URL=https://app.zibby.app

Note: The --open flag only works when uploading to cloud (requires --project or --sync flag). It will not open anything for local-only runs.

Quick Reference - Cursor Agent Commands

# ============================================
# SETUP
# ============================================

# Install cursor-agent (CI/CD only)
curl https://cursor.com/install -fsS | bash

# Patch for CI/CD
zibby ci-setup

# Get approval keys
zibby ci-setup --get-keys --save

# Setup Playwright MCP
zibby setup-playwright --headed


# ============================================
# RUN TESTS
# ============================================

# Basic run (local)
zibby run test-specs/auth/login.txt --agent=cursor

# With options and open results
zibby run test-specs/auth/login.txt \
  --agent=cursor \
  --headless \
  --collection="Auth Tests" \
  --open

# Quick open with short flag
zibby run test-specs/auth/login.txt \
  --project zby_xxx \
  -o

# CI/CD run
zibby run test-specs/auth/login.txt \
  --agent=cursor \
  --auto-approve \
  --headless \
  --sync


# ============================================
# DEBUGGING
# ============================================

# Run in headed mode (see browser)
zibby run test-specs/auth/login.txt --agent=cursor

# Run specific node
zibby run test-specs/auth/login.txt \
  --agent=cursor \
  --node=execute_live

# Resume from last session
zibby run test-specs/auth/login.txt \
  --agent=cursor \
  --node=generate_script \
  --session=last

# Check logs
tail -f ~/.cursor-agent/logs/agent.log
cat test-results/sessions/*/execute_live/events.json


# ============================================
# WORKFLOWS
# ============================================

# Download workflow
zibby workflow download --type=run_test

# Run with custom workflow
zibby run test-specs/auth/login.txt \
  --agent=cursor \
  --workflow=CustomWorkflow

# List workflows
zibby workflow list


# ============================================
# ENVIRONMENT SETUP
# ============================================

# Local development (.env)
ANTHROPIC_API_KEY=sk-ant-...  # For Claude
# No CURSOR_API_KEY needed if Cursor IDE installed

# CI/CD (.env or secrets)
CURSOR_API_KEY=your-token      # Required in CI/CD
ZIBBY_API_KEY=zby_live_...     # For cloud sync
ZIBBY_PROJECT_ID=proj_...      # Optional

Troubleshooting Reference

| Error | Solution | |-------|----------| | cursor-agent not found | Install: curl https://cursor.com/install -fsS \| bash | | CURSOR_API_KEY not set | Set env var from https://cursor.com/settings | | Tool approval required | Run zibby ci-setup | | Tests hang | Run in headed mode to see browser | | MCP connection errors | Run zibby setup-playwright --headed | | Session files missing | Check test-results/sessions/ folder | | Video not uploaded | Ensure --sync flag and ZIBBY_API_KEY set | | Browser doesn't open with --open | Check ZIBBY_ENV and frontend URL settings | | --open does nothing | Ensure upload succeeded (requires --project or --sync) | | Not a member of this project | Run zibby login to authenticate as project member | | Invalid user token | User JWT expired, run zibby login again |

License

MIT