🧠 VeriQA Test Advisor

AI-Powered Regression Test Case Advisor CLI Tool 🇮🇳 Made in India

VeriQA intelligently suggests which test cases to run based on:

• 📊 Git commits/diffs analysis
• 🤖 AI (Perplexity) suggestions
• 🗺️ Module-to-Test mapping
• ⚡ Custom CLI execution
• 📋 Enhanced Reporting (PDF & CSV formats) ✨ NEW

✨ Latest Updates

🎉 Enhanced Reporting System (v2.0)

Complete integration of simplified enhanced reporting supporting PDF and CSV formats only:

• 📊 Multi-Format Reports: Generate both PDF and CSV reports simultaneously
• 🏢 Enterprise Features: Context-specific messaging for different audiences
• 🧹 Smart Cleanup: Automatic maintenance of latest 5 reports per format
• ⚡ CLI Integration: Seamlessly integrated into main CLI without code duplication

# Generate CSV report for TCS submissions
veriqa signoff --format csv

# Generate both PDF and CSV formats
veriqa signoff --multi-format

# Clean up old reports with statistics
veriqa cleanup --stats

🎯 Why VeriQA?

| Problem | VeriQA Solution | |---------|----------------| | Test suites are growing large; running full regression is costly | Use Git diff + module mapping to detect impacted test cases | | Manual mapping of modules to tests is tedious and error-prone | Automatically generate mapping JSON using directory structures | | QA/devs may not know what test cases are relevant to a change | Use AI to suggest intelligent test cases based on commit message | | Integration with CI tools is often manual | Create a CLI that can be easily integrated into any CI/CD pipeline | | Need comprehensive reporting for stakeholders and TCS | Enhanced PDF/CSV reporting with enterprise messaging ✨ |

🚀 Quick Start

Installation

# Install globally (recommended)
npm install -g veriqa-test-advisor

# Or install locally in your project
npm install veriqa-test-advisor --save-dev

🎯 Zero-Config Setup

VeriQA provides automatic project setup for new users:

# Install and auto-setup your project
npm install -g veriqa-test-advisor
npx veriqa-setup

# This creates:
# ✅ Sample test files with @module: tags
# ✅ Configuration files (.env, playwright.config.js)  
# ✅ Project structure (tests/auth, tests/product)
# ✅ Ready-to-run examples

Basic Usage

# Get AI suggestions based on latest commit
veriqa-test-advisor --ai

# Run suggested test cases
veriqa-test-advisor --run

# Filter by tags and environment
veriqa-test-advisor --run --tag smoke --env staging

# Dry run (show what would be executed)
veriqa-test-advisor --dry-run --tag regression

📦 NPM Scripts (Auto-Added)

After setup, use these convenient commands:

npm run test:ai          # AI-powered test suggestions
npm run test:smoke       # Run smoke tests
npm run test:regression  # Run regression tests
npm run test:dry         # Dry run mode

# 🎯 Interactive Demos & Setup
npm run demo:simple      # Interactive VeriQA demo (4 modes)
npm run setup:github     # Enhanced GitHub Copilot setup

🎭 Interactive Demo Modes

# Quick overview (30 seconds)
npm run demo:simple

# Select from 4 demo modes:
# 1️⃣ Quick Overview - Essential commands  
# 2️⃣ Interactive Demo - Live project analysis
# 3️⃣ Advanced Features - Complete feature tour
# 4️⃣ Installation Guide - Step-by-step setup

📖 Command Reference

| Command | Description | |---------|-------------| | veriqa-test-advisor | Suggest test cases based on code changes | | --ai | Use AI to suggest test cases via commit message | | --run | Run only suggested test cases | | --tag <tag> | Filter tests by tag (@smoke, @regression, etc.) | | --env <env> | Pass environment variable or profile | | --dry-run | Show test plan without execution | | --verbose | Show internal debug information | | --version | Show version information | | --help | Show help message with installed dependencies |

⚙️ Setup & Configuration

1. Test Structure

Organize your tests with module annotations:

// tests/auth/login.test.js
// @module: auth

const { test, expect } = require('@playwright/test');

test('TC_Login_01 @smoke @regression - should login', async ({ page }) => {
  await page.goto('/login');
  await page.fill('#username', 'testuser');
  await page.fill('#password', 'password123');
  await page.click('#login-btn');
  await expect(page.locator('text=Welcome')).toBeVisible();
});

2. Environment Variables

Create a .env file:

# Recommended: Use your GitHub Copilot license!
GITHUB_TOKEN=your_github_token_here

# Alternative: Perplexity AI (free option)  
PERPLEXITY_API_KEY=your_perplexity_api_key
OPENAI_MODEL=sonar-pro

# VeriQA Settings
VERIQA_FRAMEWORK=playwright
VERIQA_ENV=staging

3. Supported Frameworks

VeriQA works with:

• ✅ Playwright (@playwright/test)
• ✅ CodeceptJS with Playwright helper
• 📊 Allure reporting (automatic)

🤖 AI Integration

VeriQA supports multiple AI providers for intelligent test suggestions:

🚀 GitHub Copilot Integration (Recommended)

Use your existing GitHub Copilot license for enhanced AI suggestions:

# Setup: Add GITHUB_TOKEN to .env (with repo, copilot scopes)
# Get token: https://github.com/settings/tokens

# AI will use GitHub Copilot to analyze your commits
veriqa-test-advisor --ai

🔄 Perplexity AI (Free Alternative)

Free option using Perplexity AI:

# Setup: Add PERPLEXITY_API_KEY to .env
# Get free key: https://www.perplexity.ai/

# Example output:
# 🤖 AI Suggested Regression Test Cases:
# - TC_Login_01.spec.js (reason: login functionality changes)
# - TC_2FA_01.spec.js (reason: authentication flow affected)
# - TC_Registration_test.js (reason: user management updates)

🗺️ Module Mapping

VeriQA automatically generates mappings/module_test_map.json:

{
  "auth": [
    "tests/auth/TC_Login_01.spec.js",
    "tests/auth/TC_2FA_01.spec.js"
  ],
  "product": [
    "tests/product/TC_Search.spec.js"
  ]
}

📊 Advanced Features

✨ Enhanced Reporting System (NEW)

VeriQA now includes comprehensive enterprise-ready reporting with PDF and CSV formats:

# Generate CSV report for TCS submissions and Excel analysis
veriqa signoff --format csv

# Generate PDF report for stakeholders and management
veriqa signoff --format pdf

# Generate both formats simultaneously
veriqa signoff --multi-format

# Custom filename with date
veriqa signoff --format csv --filename "tcs-submission-$(date +%Y%m%d)"

🏢 Enterprise Features

• 📊 Smart Multi-Format: Generate PDF and CSV simultaneously with --multi-format
• 🧹 Automatic Cleanup: Maintains latest 5 reports per format automatically
• 🎯 Context Messaging: Format-specific messages for different audiences
• 📈 Statistics: View report statistics with --stats option

📋 Report Commands

# Report cleanup with statistics
veriqa cleanup --stats --max 3

# Preview cleanup without deletion
veriqa cleanup --dry-run

# Generate quarterly review reports
veriqa signoff --multi-format --filename "Q1-2024-review"

📊 Report Formats

| Format | Use Case | Audience | Command | |--------|----------|----------|---------| | CSV | TCS submissions, Excel analysis | Analysts, QA Teams | --format csv | | PDF | Stakeholder presentations | Management, Executives | --format pdf | | JSON | CI/CD integration, automation | Development Teams | --format json | | Console | Daily development workflow | Developers | --format console |

K6-Style Test Summary

After test execution, get detailed metrics:

running (veriqa-advisor) ...
wall time:         2m 34.2s
tests:             25
✓ passed:          23 (92.0%)
✗ failed:          2 (8.0%)
avg duration:      3.4s
p50 | p90:         2.1s | 8.7s
p95 | p99:         12.3s | 18.9s
RPS:               0.16

by framework:
  playwright       20 (80.0%)
  codeceptjs       5 (20.0%)

by tag:
  smoke           15 (60.0%)
  regression      10 (40.0%)

CI/CD Integration

# GitHub Actions example
- name: Run Smart Regression Tests
  run: |
    npx veriqa-test-advisor --run --tag smoke
    
# GitLab CI example  
test:regression:
  script:
    - npx veriqa-test-advisor --ai --verbose
    - npx veriqa-test-advisor --run --env $CI_ENVIRONMENT_NAME

🎨 Examples

Basic Workflow

# 1. Make code changes
git add . && git commit -m "fix: update login validation logic"

# 2. Get AI suggestions
veriqa-test-advisor --ai

# 3. Run suggested tests
veriqa-test-advisor --run --tag smoke

# 4. Run full regression if needed
veriqa-test-advisor --run --tag regression

Advanced Usage

# Debug mode with verbose output
veriqa-test-advisor --ai --verbose

# Dry run to see what would be executed
veriqa-test-advisor --dry-run --tag "smoke|regression"

# Environment-specific testing
VERIQA_ENV=staging veriqa-test-advisor --run --tag smoke

🔧 Configuration Files

Playwright Config

// playwright.config.js
module.exports = {
  testDir: 'tests',
  reporter: [
    ['list'],
    ['allure-playwright']  // Required for VeriQA metrics
  ]
};

CodeceptJS Config

// codecept.conf.js  
exports.config = {
  tests: 'tests/**/*.js',
  plugins: {
    allure: {
      enabled: true,
      require: '@codeceptjs/allure-legacy'
    }
  }
};

📈 Business Value

• 📉 Reduce CI/CD costs by 60-80% (run only relevant tests)
• ⚡ Faster feedback for developers (smart test selection)
• 🎯 Higher test coverage confidence (AI-driven suggestions)
• 🔄 Easy integration with existing pipelines
• 📊 Better insights with detailed test analytics

🛠️ Troubleshooting

Quick Diagnostics

# Comprehensive project analysis & diagnosis
npx veriqa-troubleshoot

# Auto-fix detected issues (config naming, imports, etc.)
npx veriqa-troubleshoot --fix

# Validate complete installation
npx veriqa-validate

Advanced Troubleshooting

Our enhanced troubleshoot system automatically detects and fixes:

✅ Config File Naming Issues

• playwright.config → playwright.config.js
• codecept.config.js → codecept.conf.js
• cypress.config → cypress.config.js
• Package.json → package.json

✅ Framework Compatibility

• Multi-framework conflict detection
• Port conflict warnings
• Dependency validation

✅ Environment Configuration

• Missing API keys detection
• .env file validation
• Smart recommendations

Common Issues

❌ "ReferenceError: devices is not defined"

# Fix Playwright config devices import
npx veriqa-troubleshoot --fix
# Or manually add: const { defineConfig, devices } = require('@playwright/test');

❌ "Command not found: veriqa-test-advisor"

# Reinstall globally
npm uninstall -g veriqa-test-advisor
npm install -g veriqa-test-advisor@latest

❌ "No tests found"

# Check project setup
npx veriqa-validate
# Regenerate mapping
veriqa-test-advisor --run --verbose

❌ "AI suggestions failed"

# Check API key setup
cat .env | grep PERPLEXITY_API_KEY
# Get your free API key at: https://perplexity.ai

Get Help

• 📖 Installation Guide
• 🔧 Troubleshooting Tool
• 🐛 Report Issues

📈 Business Value

• 📉 Reduce CI/CD costs by 60-80% (run only relevant tests)
• ⚡ Faster feedback for developers (smart test selection)
• 🎯 Higher test coverage confidence (AI-driven suggestions)
• 🔄 Easy integration with existing pipelines
• 📊 Better insights with detailed test analytics

🤝 Contributing

We welcome contributions! Please see our Contributing Guide.

📄 License

MIT © Kapil Rathore

🔗 Links

• 📦 npm package
• 🐙 GitHub repository
• 📖 Documentation
• 💬 Discord Community

👨‍💻 Author

Kapil Rathore - Creator & Maintainer

• 🐙 GitHub: @kapil971390
• 📧 Email: [email protected]
• 💼 LinkedIn: Connect with me

🔄 Version Management

VeriQA includes automatic version checking and update notifications:

Version Commands

# Check for updates
veriqa-version --check

# Show detailed version info
veriqa-version --info

# View version history
npm run version:info

Automatic Updates

• 🔍 Auto-check: Checks for updates every 24 hours (non-intrusive)
• 📢 Smart notifications: Shows update alerts only once per version
• 📊 Usage analytics: Anonymous usage tracking for better development
• 🎯 Update urgency: Color-coded notifications (🚨 Major, ✨ Minor, 🔧 Patch)

Update Service (Optional)

# Start background update service
veriqa-update-service --start

# Check service status
veriqa-update-service --status

# Manual update check
veriqa-update-service --check

Privacy & Analytics

VeriQA collects anonymous usage data to improve the tool:

• ✅ Anonymous: No personal information collected
• ✅ Local: Data stored locally on your machine
• ✅ Minimal: Only feature usage and version information
• ✅ GDPR Compliant: Clear and export your data anytime

# View your analytics data
node -e "console.log(JSON.stringify(require('./node_modules/veriqa-test-advisor/src/versionAnalytics').prototype.exportAnalytics(), null, 2))"

# Clear analytics data
node -e "new (require('./node_modules/veriqa-test-advisor/src/versionAnalytics'))().clearAnalytics()"

🤝 Contributing

Contributions are welcome! Please read CONTRIBUTING.md for details on our code of conduct and the process for submitting pull requests.

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

Copyright (c) 2025 Kapil Rathore