@openqa/cli
v1.3.4
Published
Autonomous QA testing agent powered by Orka.js
Downloads
2,245
Maintainers
Readme
OpenQA is a truly autonomous QA testing agent that thinks, codes, and executes tests like a senior QA engineer. It analyzes your application, generates its own tests, creates custom agents, and continuously improves.
✨ Features
🧠 Autonomous Brain
� Autonomous Brain
- Self-thinking - Analyzes your app and decides what to test
- Self-coding - Generates unit, functional, E2E, security, and regression tests
- Self-improving - Learns from results and adapts strategy
- Dynamic Agents - Creates specialized agents on-the-fly based on needs
� Simple Configuration
- Describe your SaaS - Just provide name, description, and URL
- Add directives - Optional instructions to guide the agent
- Connect repo - Optional: clone and analyze your codebase
- That's it! - The agent figures out the rest
🔗 Git Integration
- GitHub/GitLab - Auto-detect merges on main branch
- CI/CD Listener - Trigger tests after successful deployments
- Auto Issues - Creates issues for critical bugs found
🧪 Test Generation
- Unit Tests - Isolated function/component tests
- Functional Tests - User workflow tests
- E2E Tests - Complete user journey tests
- Security Tests - SQL injection, XSS, auth bypass
- Regression Tests - Verify bug fixes
- Performance Tests - Load times, resource usage
🚀 Quick Start
One-Line Installation
curl -fsSL https://openqa.orkajs.com/install.sh | bashConfigure Your SaaS (3 lines!)
# Configure your application
curl -X POST http://localhost:3000/api/saas-config/quick \
-H "Content-Type: application/json" \
-d '{
"name": "My SaaS App",
"description": "E-commerce platform with user auth, product catalog, and checkout",
"url": "https://my-app.com"
}'
# Start autonomous testing
curl -X POST http://localhost:3000/api/brain/runThat's it! OpenQA will:
- Analyze your application
- Generate appropriate tests
- Create specialized agents as needed
- Execute tests and report findings
- Create GitHub issues for critical bugs
Manual Installation
git clone https://github.com/orka-js/openqa.git
cd openqa
npm install && npm run build
cp .env.example .env # Add your LLM API key
npm start📖 Usage
Start OpenQA
# Start in foreground (see logs)
openqa start
# Start in daemon mode (background)
openqa start --daemonAccess Web Interfaces
Once started, open your browser:
- DevTools: http://localhost:3000 - Monitor agent activity in real-time
- Kanban: http://localhost:3000/kanban - View and manage QA tickets
- Config: http://localhost:3000/config - Configure OpenQA settings
CLI Commands
# Check status
openqa status
# View logs
openqa logs
openqa logs --follow
# Configuration
openqa config list
openqa config get llm.provider
openqa config set saas.url https://your-app.com
# Stop agent
openqa stop⚙️ Configuration
Environment Variables
# LLM Configuration
LLM_PROVIDER=openai # openai, anthropic, or ollama
OPENAI_API_KEY=sk-xxx
ANTHROPIC_API_KEY=
OLLAMA_BASE_URL=http://localhost:11434
# Target SaaS Application
SAAS_URL=https://your-app.com
SAAS_AUTH_TYPE=none # none, basic, or session
SAAS_USERNAME=
SAAS_PASSWORD=
# GitHub Integration (Optional)
GITHUB_TOKEN=ghp_xxx
GITHUB_OWNER=your-username
GITHUB_REPO=your-repo
# Agent Behavior
AGENT_INTERVAL_MS=3600000 # 1 hour between test sessions
AGENT_MAX_ITERATIONS=20 # Max actions per session
AGENT_AUTO_START=true # Start testing automatically
# Web UI
WEB_PORT=3000
WEB_HOST=0.0.0.0
# Database
DB_PATH=./data/openqa.dbWeb-based Configuration
Prefer using the web interface at http://localhost:3000/config for easier configuration.
🧠 How the Brain Works
The Thinking Loop
┌───────────┐ ┌───────────┐ ┌───────────┐
│ ANALYZE │ →→→ │ THINK │ →→→ │ ACT │
└───────────┘ └───────────┘ └───────────┘
│ │ │
│ │ │
└──────────────────┴──────────────────┘
LEARN- ANALYZE - Understands your app (description, URL, code if available)
- THINK - Decides what tests are needed, what agents to create
- ACT - Generates tests, creates agents, executes tests
- LEARN - Reviews results, adjusts strategy, repeats
What It Generates
| Test Type | Description | Example |
|-----------|-------------|----------|
| Unit | Isolated function tests | test_validateEmail() |
| Functional | User workflow tests | test_loginFlow() |
| E2E | Complete journeys | test_purchaseCheckout() |
| Security | Vulnerability tests | test_sqlInjection() |
| Regression | Bug fix verification | test_issue123_fix() |
| Performance | Speed/load tests | test_pageLoadTime() |
Dynamic Agent Creation
The Brain creates specialized agents based on what it discovers:
🧠 Brain: "I see this app has a checkout flow. Let me create a specialist."
↓
🤖 Creates: "Checkout Flow Tester" agent with custom prompt
↓
🧪 Agent generates: 5 tests for cart, payment, confirmation
↓
✅ Executes tests, reports 2 bugs found📊 Web Interfaces
DevTools Dashboard
Real-time monitoring:
- Current agent status
- Live action logs
- Test session history
- Bugs detected
- Screenshots captured
Kanban Board
Manage QA findings:
- Backlog - Future test ideas
- To Do - Bugs to fix
- In Progress - Being worked on
- Done - Resolved
Drag & drop tickets between columns.
Configuration Panel
Simple configuration:
- SaaS Info - Name, description, URL
- Repository - Optional GitHub/GitLab URL for code analysis
- Directives - Custom instructions for the agent
- LLM Settings - Provider, API key, model
- Git Listener - Auto-test on merge/deploy
🔧 Advanced Usage
Full Configuration
# Configure with all options
curl -X POST http://localhost:3000/api/saas-config \
-H "Content-Type: application/json" \
-d '{
"name": "My E-commerce App",
"description": "Online store with user accounts, product catalog, shopping cart, and Stripe checkout",
"url": "https://my-store.com",
"repoUrl": "https://github.com/myorg/my-store",
"directives": [
"Focus on the checkout flow - it has had bugs before",
"Test with both logged-in and guest users",
"Check that discount codes work correctly",
"Verify email notifications are sent"
],
"auth": {
"type": "session",
"credentials": { "username": "[email protected]", "password": "testpass" }
}
}'Add Directives On-the-fly
# Add a new directive
curl -X POST http://localhost:3000/api/saas-config/directive \
-H "Content-Type: application/json" \
-d '{"directive": "Pay special attention to the new refund feature"}'Connect Repository for Code Analysis
# The Brain will clone and analyze your code
curl -X POST http://localhost:3000/api/saas-config/repository \
-H "Content-Type: application/json" \
-d '{"url": "https://github.com/myorg/my-app"}'Generate Specific Tests
# Generate a security test
curl -X POST http://localhost:3000/api/brain/generate-test \
-H "Content-Type: application/json" \
-d '{
"type": "security",
"target": "Login form SQL injection",
"context": "The login form at /login accepts email and password"
}'
# Generate a functional test
curl -X POST http://localhost:3000/api/brain/generate-test \
-H "Content-Type: application/json" \
-d '{
"type": "functional",
"target": "User registration flow"
}'Create Custom Agent
# Let the Brain create a specialized agent
curl -X POST http://localhost:3000/api/brain/create-agent \
-H "Content-Type: application/json" \
-d '{"purpose": "Test the multi-step checkout process with various payment methods"}'Analyze Before Running
# Get analysis and suggestions without running tests
curl -X POST http://localhost:3000/api/brain/analyze
# Response:
# {
# "understanding": "E-commerce app with user auth, catalog, cart, checkout",
# "suggestedTests": ["Test cart persistence", "Test checkout validation", ...],
# "suggestedAgents": ["Payment Flow Specialist", "Inventory Checker"],
# "risks": ["Cart race conditions", "Payment double-charge"]
# }Docker Deployment
docker-compose up -d🛠️ Development
# Install dependencies
npm install
# Run in development mode
npm run dev
# Build
npm run build
# Run tests
npm test📝 Architecture
openqa/
├── agent/
│ ├── brain/ # 🧠 Autonomous thinking & test generation
│ ├── tools/ # Browser, GitHub, Kanban tools
│ ├── config/ # SaaS configuration
│ └── webhooks/ # Git listener (GitHub/GitLab)
├── cli/ # CLI commands
├── database/ # SQLite database
└── data/
├── workspace/ # Cloned repos
└── generated-tests/ # Tests created by Brain🔌 API Reference
SaaS Configuration
GET /api/saas-config- Get current configPOST /api/saas-config- Full configurationPOST /api/saas-config/quick- Quick setup (name, description, url)POST /api/saas-config/directive- Add directivePOST /api/saas-config/repository- Set repo URLPOST /api/saas-config/local-path- Set local path
Brain Control
POST /api/brain/analyze- Analyze app, get suggestionsPOST /api/brain/run- Start autonomous sessionPOST /api/brain/generate-test- Generate specific testPOST /api/brain/create-agent- Create custom agentPOST /api/brain/run-test/:id- Execute a test
Data
GET /api/status- Agent statusGET /api/tests- Generated testsGET /api/dynamic-agents- Created agentsGET /api/sessions- Test sessionsGET /api/bugs- Found bugsGET /api/kanban/tickets- Kanban board
WebSocket (ws://localhost:3000)
test-generated- New test createdagent-created- New agent createdtest-started/completed- Test lifecyclethinking- Brain's current thoughtanalysis-complete- Analysis finishedsession-complete- Session finishedgit-merge- Merge detectedgit-pipeline-success- Deploy detected
🤝 Contributing
Contributions welcome! Please read CONTRIBUTING.md first.
📄 License
MIT
🙏 Credits
Built with:
- Orka.js - AI framework
- Playwright - Browser automation
- Next.js - Web interface
- SQLite - Database
Made with ❤️ by the Orka.js team
