QA Guardian Command Line Interface

Trigger and monitor test suite executions from the command line.

Installation

npm install -g qaguardian

Or use directly with npx:

npx qaguardian --tags smoke

Quick Start

# Set your API key
export QAGUARDIAN_API_KEY=your-api-key-here

# Run suites with specific tags
npx qaguardian --tags auth,login

# Exclude certain tags
npx qaguardian --tags regression --exclude-tags slow,flaky

# Run all suites
npx qaguardian --all

# Fire and forget (no polling)
npx qaguardian --tags smoke --no-wait

Gate a pull request on the suite at its head commit (tests in your own repository — see CI/CD below for why $GITHUB_SHA is the wrong value on pull_request events):

npx qaguardian --all --ref "$PR_HEAD_SHA"

Options

• --tags <tags> - Comma-separated tags to match (e.g., "auth,ci,smoke")
• --exclude-tags <tags> - Comma-separated tags to exclude (e.g., "slow,flaky")
• --all - Run all suites (combine with --exclude-tags to filter)
• --match-mode <mode> - Tag matching mode: "any" (OR logic, default) or "all" (AND logic)
• --ref <git-ref> - Only applies when your tests live in your own repository (no effect for Guardian-hosted flows): run the suite as it exists at this branch, tag or commit, and post a "Guardian release gate" commit status on that exact commit. On a pull request, this must be the PR's head commit SHA, not your CI provider's default "current commit" variable — see CI/CD. With --ref, the CLI's exit code follows that release gate rather than the triggered runs' own results — see Release gate exit code below.
• --timeout <seconds> - Only with --ref: max seconds to wait for the release gate to resolve once the triggered runs are terminal (default: 900 / 15 minutes). Guardian may run one automatic confirmation re-run of a failure before the gate resolves, so this needs to cover that re-run too, not just the original run.
• --allow-empty - Exit 0 with a warning when zero flows ran (no match, or every matching flow is disabled), instead of failing with exit code 2. Also settable with QAGUARDIAN_ALLOW_EMPTY=1.
• --no-wait - Trigger and exit immediately without waiting for results
• --webhook-url <url> - Custom webhook URL for notifications on completion
• --notify <service> - Notify via service: slack, discord, google-chat, or teams
• --api-url <url> - Custom API Gateway URL (default: https://api.qaguardian.com)

Environment Variables

• QAGUARDIAN_API_KEY - (Required) Your QA Guardian API key
• QAGUARDIAN_API_URL - (Optional) API Gateway URL (defaults to https://api.qaguardian.com)
• QAGUARDIAN_ALLOW_EMPTY - (Optional) 1/true/yes is the same as --allow-empty

Examples

Tag-based execution with wait

export QAGUARDIAN_API_KEY=guardians-primary-qag-xxxxxxxxxxxxxxxx
npx qaguardian --tags auth,ci

Exclude certain tags

npx qaguardian --tags regression --exclude-tags slow,flaky

Run everything except auth suites

npx qaguardian --all --exclude-tags auth

Fire and forget

npx qaguardian --tags smoke --no-wait

AND logic (all tags required)

npx qaguardian --tags auth,login,ui --match-mode all

Notifications

Receive notifications when tests complete:

# Custom webhook URL
npx qaguardian --tags regression --webhook-url https://my-service.com/webhook

# Slack notification
npx qaguardian --tags smoke --notify slack

# Discord notification
npx qaguardian --tags auth --notify discord

# Google Chat notification
npx qaguardian --tags ci --notify google-chat

# Microsoft Teams notification
npx qaguardian --tags regression --notify teams

Local development

cd qaguardian
npm install
npm link
export QAGUARDIAN_API_KEY=your-api-key
npx qaguardian --tags smoke --api-url http://localhost:8080

Release gate exit code (with --ref)

Without --ref, the CLI's exit code is a direct read of the flow runs it triggered: any failure exits 1.

With --ref, Guardian may run one automatic confirmation re-run of a failure before deciding the "Guardian release gate" commit status for that commit — a single failure isn't necessarily a real regression, and the gate stays in a pending "confirming" state until the re-run resolves it. So with --ref, once the triggered runs are all terminal, the CLI instead polls the release gate itself and follows its verdict:

• gate success → exit 0 (this includes a failure that was confirmed flaky by a passing re-run, and a failed gate a Guardian has manually overridden — the CLI prints "Overridden by a Guardian" in that case)
• gate failure or error → exit 1 (this includes a failure that was reproduced by the confirmation re-run)
• gate still pending after --timeout seconds → exits non-zero with a message explaining the gate never resolved in time

The CLI prints the gate's description and a link to the run/gate page either way.

CI/CD

The CLI works from any CI provider — it makes one HTTPS request to trigger the run and polls the API for the result, so a job does not need to check out your repository. It does need the exact commit SHA under test, and on a pull_request-style trigger that is not the same as your provider's default "current commit" variable.

Why $GITHUB_SHA doesn't gate a pull request: on GitHub Actions, pull_request events check out and set GITHUB_SHA/github.sha to a throwaway merge commit between your branch and the base branch — it is never part of the PR's own commit history. Guardian posts the "Guardian release gate" status on the commit you pass with --ref, so posting it on that merge commit produces a status GitHub cannot attach to the PR, and a required check configured for it will never be satisfied. Use github.event.pull_request.head.sha instead. (push events don't have this problem — github.sha is the actual pushed commit.)

GitHub Actions

Store QAGUARDIAN_API_KEY in GitHub Secrets.

name: QA Guardian release gate

on:
  pull_request:
  push:
    branches: [main]

permissions: {}

jobs:
  guardian-gate:
    runs-on: ubuntu-latest
    steps:
      - name: Resolve the commit to gate
        id: gate-sha
        # pull_request: use the PR's head commit, not $GITHUB_SHA (a throwaway
        # merge commit that isn't part of the PR's history).
        # push: $GITHUB_SHA is already the real pushed commit.
        run: |
          if [ "${{ github.event_name }}" = "pull_request" ]; then
            echo "sha=${{ github.event.pull_request.head.sha }}" >> "$GITHUB_OUTPUT"
          else
            echo "sha=${{ github.sha }}" >> "$GITHUB_OUTPUT"
          fi

      - name: Set up Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Run QA Guardian
        env:
          QAGUARDIAN_API_KEY: ${{ secrets.QAGUARDIAN_API_KEY }}
        run: npx qaguardian --all --ref "${{ steps.gate-sha.outputs.sha }}"

Make it required: Settings → Branches → Branch protection rule (on your default branch) → Require status checks to pass → select Guardian release gate. GitHub only offers a status in that list once it has been posted at least once, so run the workflow on a PR first, then go back and require it.

GitLab CI

Store QAGUARDIAN_API_KEY in GitLab CI/CD variables (masked). Requires merge request pipelines to be enabled for the project.

qaguardian-release-gate:
  stage: test
  image: node:20-slim
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
  script:
    # CI_MERGE_REQUEST_SOURCE_BRANCH_SHA is the source branch's actual head
    # commit even if "pipelines for merged results" is enabled, where
    # CI_COMMIT_SHA would otherwise be a synthetic merge commit instead.
    - export GATE_SHA="${CI_MERGE_REQUEST_SOURCE_BRANCH_SHA:-$CI_COMMIT_SHA}"
    - npx qaguardian --all --ref "$GATE_SHA"

Make it required: Settings → Merge requests → Merge checks → Pipelines must succeed. Unlike GitHub, GitLab merge checks gate on overall pipeline success rather than a named status, so this job failing is what blocks the merge — no separate check needs selecting.

Generic shell

Any CI system that can export an env var and run Node works the same way:

export QAGUARDIAN_API_KEY=your-api-key-here
# Use the exact commit your provider is testing for this PR/MR — not
# necessarily its default "current commit" variable. See the notes above.
npx qaguardian --all --ref "<pr-or-mr-head-commit-sha>"

Exit Codes

• 0 - All flow runs completed with a verified pass (status=completed and a passing result), or no flow matched and you passed --allow-empty (a warning is printed).
• 1 - Any flow run failed, timed out, was cancelled, could not be polled (a persistent API error), or reported a status/result this CLI version doesn't recognize as a pass. The gate fails closed: only a run that is unambiguously status=completed with a passing result (passed, skipped, or no result) is ever reported as green: anything else -- including a brand-new backend status added after this CLI version shipped -- fails the gate rather than silently passing it. Polling that exceeds the configured timeout also exits 1.
• 2 - Zero flows ran (no flow matched your filters, or every matching flow is disabled), so nothing was tested. The CLI prints the coordinator's message and fails the gate. Pass --allow-empty to get exit 0 instead. Applies with --no-wait too. If flows matched but none could be triggered (all dispatches failed, or all were only queued), the CLI exits 1 regardless of --allow-empty.

API Key

Get your API key from the QA Guardian dashboard at: https://qaguardian.com/settings/api-keys

Polling Behavior

By default, the CLI polls the API every 30 seconds to check for completion. You can observe:

• Real-time test count updates
• Pass/fail counts as tests complete
• Automatic exit with appropriate code when all suites finish

Use --no-wait to skip polling and exit immediately after triggering.

Notifications

The CLI can send notifications when test suites complete. Choose one of:

Webhook Notifications

Send results to any HTTP endpoint:

qaguardian --tags regression --webhook-url https://my-service.com/webhook

The webhook receives a POST request with execution results.

Platform Notifications

Integrated notification support for popular chat and communication platforms:

• Slack: --notify slack (requires Slack integration configured in QA Guardian)
• Discord: --notify discord (requires Discord integration configured)
• Google Chat: --notify google-chat (requires Google Chat integration configured)
• Microsoft Teams: --notify teams (requires Teams integration configured)

Example:

qaguardian --tags smoke --notify slack

Notifications are sent when all tests complete, with summary of pass/fail counts.

Publishing

Requires maintainer access on npmjs.com/package/qaguardian.

Steps

1. Bump the version in package.json
2. Commit: git commit -m "vX.Y.Z"
3. Log in to npm (first time or if session expired):

   npm login

4. Publish:

   npm publish --access public

   npm will open a browser tab to complete authentication (email OTP).

Automated publishing (CI)

CI publishes with npm trusted publishing (OIDC): npmjs.com trusts this project's .gitlab-ci.yml, so no npm token is stored anywhere. Bump version in package.json and push to main (or play the manual publish-from-dev job on dev). The job runs on a GitLab.com shared runner because npm only accepts OIDC from GitLab-hosted runners.

License

MIT