@automatify-au/cli

Minimal Forge-first CLI for Automatify Jira TestOps.

Current product stance:

• Jira UI is the primary day-to-day surface for most users.
• The CLI is the primary admin and automation surface for setup, ingestion, diagnostics, reporting, and CI.
• POC audience: operators, CI, internal support, and customer admins running controlled setup/import flows.
• Future customer-admin packaging is expected before any broader distribution.
• Any future all-user CLI requires a different auth/onboarding model than the current project-scoped admin key transport.

Command-group note:

• The groupings below describe intended usage and complexity only.
• They are not a security boundary in the current transport model.
• A valid project CLI token is not yet scoped per command group.

Architecture boundary (must hold)

• Thin client only.
• No separate backend.
• No duplicate domain/business logic in CLI.
• CLI commands must call shared Forge contracts (resolver/webtrigger paths) as they are introduced.
• Focused evidence commands may call Jira Cloud REST APIs directly when the feature is specifically Jira-native attachment/comment work.

MVP non-goals

• No separate backend.
• No duplicate domain/business logic in CLI.
• No replacement of Jira UI as a primary product surface.
• No backend-assisted/hybrid logic in this phase.

Install and run

From npm (published package)

npm install -g @automatify-au/cli
automatify --help
automatify testops --help

From repository (development)

pnpm install
pnpm --filter @automatify-au/cli build
pnpm --filter @automatify-au/cli bundle

This produces a single self-contained bundle at dist/automatify.cjs (≈125 KB) with @automatify/shared-types inlined. No runtime dependencies.

Run built CLI:

automatify testops --help            # if linked globally (npm link)
node apps/cli/dist/automatify.cjs --help  # direct node run

Dev run (for local iteration):

pnpm --filter @automatify-au/cli run cli:dev -- --help

Bundle and publish

cd apps/cli
npm test                      # optional but recommended local preflight
npm pack --dry-run            # inspect final tarball contents
npm publish --dry-run --access public  # simulate publish against npm
npm publish --access public   # publish to npm

Publish notes:

• prepack now rebuilds dist/automatify.js automatically, so npm pack and npm publish do not rely on a stale bundle.
• prepublishOnly runs the CLI test suite and then rebuilds the bundle before a real publish.
• Current package target is @automatify-au/cli; verify npm auth with npm whoami if needed before publishing.

Required configuration

Minimum:

• JIRA_BASE_URL
• At least one of JIRA_PROJECT_KEY or JIRA_ISSUE_KEY
• TESTOPS_FORGE_ENDPOINT for commands that call Forge contracts (doctor, ingest, run, sync)
• TESTOPS_FORGE_AUTH_TOKEN from the one-time token shown in Forge Operations -> CLI access

Local config helper:

automatify testops config set baseUrl https://automatify-com-au.atlassian.net
automatify testops config set projectKey DEV
automatify testops config set forgeEndpoint "<webtrigger-url>"
printf "%s" "$TESTOPS_FORGE_AUTH_TOKEN" | automatify testops config set forgeAuthToken --stdin

forgeAuthToken local storage currently uses macOS Keychain. On Windows, Linux, and CI, keep the token in TESTOPS_FORGE_AUTH_TOKEN instead. .testops-cli.json should store non-secret values only.

Optional:

• TESTOPS_FORGE_TIMEOUT_MS
• TESTOPS_FORGE_MAX_RETRIES
• TESTOPS_FORGE_RETRY_DELAY_MS
• TESTOPS_AUTH_MODE, JIRA_EMAIL, JIRA_API_TOKEN (when auth mode requires)

Command groups (MVP)

All TestOps commands are invoked as automatify testops <command>:

Core browse commands

• bdd
  • automatify testops bdd scenarios show --id <SCENARIO_ID>
  • automatify testops bdd scenarios show --id <SCENARIO_ID> --format feature
  • automatify testops bdd scenarios show --id <SCENARIO_ID|SC-4> --feature
  • automatify testops bdd scenarios export --id <SCENARIO_ID|SC-4> --output-dir ./scenario-export
  • automatify testops bdd scenarios export --all --output-dir ./scenario-export
  • automatify testops bdd scenarios import --file ./scenario-export/SC-4.feature
  • automatify testops bdd scenarios import --file ./scenario-export/SC-4.feature --dry-run
  • automatify testops bdd scenarios import --input-dir ./scenario-export
  • automatify testops bdd scenarios import --input-dir ./scenario-export --dry-run
  • automatify testops bdd features export --output-dir ./features-export
  • automatify testops bdd features export --output-dir ./features-export --zip
  • automatify testops bdd features export --output-dir ./features-export --feature-id <FEATURE_ID>
• runs
  • automatify testops runs show --id <RUN_ID>
• config
  • automatify testops config show
  • automatify testops config validate
• doctor
  • automatify testops doctor check
  • automatify testops doctor check --json
• allure
  • automatify allure doctor
  • automatify allure doctor --json
  • automatify allure doctor --issue-key ABC-123
  • automatify allure open ./allure-report.zip
  • automatify allure open ./allure-report.zip --no-open --port 8080

Advanced browse commands

• cases
  • automatify testops cases list
  • automatify testops cases show --key <TEST_CASE_KEY>
  • automatify testops cases bdd list --key <TEST_CASE_KEY>
  • automatify testops cases runs list --key <TEST_CASE_KEY>
• suites
  • automatify testops suites list
  • automatify testops suites show --key <SUITE_KEY>
  • automatify testops suites cases list --key <SUITE_KEY>

Setup, ingestion, and CI commands

• ingest
  • automatify testops ingest feature --file <path> or --stdin
• run
  • automatify testops run upload --file <path> or --stdin
• allure
  • automatify testops allure upload ABC-123 ./allure-report.zip --dry-run
  • automatify testops allure upload ABC-123 ./allure-report.zip
  • automatify testops allure download ABC-123 --output-dir ./downloads
  • automatify testops allure open ./allure-report.zip
• auto
  • automatify testops auto --dry-run
  • automatify testops auto

Allure Evidence for Jira

Purpose:

• upload an already generated Allure HTML report ZIP to a Jira issue as evidence.
• download an uploaded Allure HTML report ZIP from a Jira issue for local inspection.
• parse widgets/summary.json from the ZIP and add a concise Jira comment with result counts.
• keep the flow Java-free and backend-free; this command does not generate Allure reports from raw allure-results.

Supported ZIP layouts:

• widgets/summary.json
• allure-report/widgets/summary.json
• any nested path ending in /widgets/summary.json

Credentials:

export JIRA_SITE="https://example.atlassian.net"
export JIRA_EMAIL="[email protected]"
export JIRA_API_TOKEN="<atlassian-api-token>"

JIRA_BASE_URL and existing .testops-cli.json baseUrl are also accepted for the site URL. CLI flags take precedence:

automatify testops allure upload ABC-123 ./allure-report.zip \
  --site https://example.atlassian.net \
  --email [email protected] \
  --api-token "$JIRA_API_TOKEN"

Dry run:

automatify testops allure upload ABC-123 ./allure-report.zip --dry-run

Upload without a Jira comment:

automatify testops allure upload ABC-123 ./allure-report.zip --no-comment

Normal output prints the Jira issue, attachment filename, attachment id, and attachment URL when Jira returns them.

Machine-readable output:

automatify testops allure upload ABC-123 ./allure-report.zip --dry-run --json

After a real upload, JSON output includes Jira attachment metadata when Jira returns it:

{
  "issueKey": "ABC-123",
  "uploaded": true,
  "commentAdded": true,
  "dryRun": false,
  "summary": {
    "total": 128,
    "passed": 120,
    "failed": 5,
    "broken": 1,
    "skipped": 2,
    "unknown": 0
  },
  "attachment": {
    "filename": "allure-report.zip",
    "id": "10001",
    "self": "https://example.atlassian.net/rest/api/3/attachment/10001",
    "content": "https://example.atlassian.net/rest/api/3/attachment/content/10001",
    "size": 4096
  }
}

Download the latest Allure report ZIP attachment from a Jira issue:

automatify testops allure download ABC-123 --output-dir ./downloads

Download selection rules:

• --attachment-id <id> downloads one exact Jira attachment.
• --filename <name> downloads the newest attachment with that filename.
• without either flag, the CLI chooses the newest allure-report*.zip attachment, then falls back to the newest .zip attachment.
• existing local files are not overwritten unless --force is passed.

Download dry run / JSON:

automatify testops allure download ABC-123 --dry-run --json

Allure preflight:

automatify allure doctor

automatify allure doctor checks:

• Jira site/email/API token resolution for direct Allure upload/download.
• optional Jira issue access only when --issue-key is passed.
• unzip availability for manual local ZIP inspection.
• Java availability as a warning-only optional check for future raw allure-results workflows.
• Allure CLI availability as a warning-only optional check.

Generated Allure HTML ZIP upload/download/open does not require Java or the Allure CLI.

Optional live Jira issue check:

automatify allure doctor --issue-key ABC-123

Open a generated Allure HTML report locally

automatify allure open ./allure-report.zip

automatify allure open <zipPath> and automatify testops allure open <zipPath> extract the report into a temporary directory, start a Java-free Node static HTTP server on an ephemeral port, and open the URL in your default browser. This is useful after downloading an Allure ZIP from Jira.

Options:

• --port <n> fixed port (default 0 for an OS-assigned free port).
• --host <h> bind host (default 127.0.0.1).
• --extract-to <dir> extract into a chosen directory; the directory is preserved on shutdown.
• --keep preserves the auto-created temp extract directory on shutdown.
• --no-open skips browser launch for CI/headless checks.
• --json, --dry-run, --verbose.

The server stays alive until Ctrl+C / SIGTERM. URL path traversal and malformed paths are rejected.

Optional gated maintenance commands

• sync
  • enable with TESTOPS_ENABLE_SYNC=1
  • automatify testops sync status
  • automatify testops sync reconcile --confirm RECONCILE

automatify testops auto (MVP)

Purpose:

• deterministic onboarding/import helper for local and CI workflows.
• thin orchestration only; domain logic stays in Forge contracts/services.

Supported commands:

• automatify testops auto --dry-run
• automatify testops auto

Required env/config for real upload mode:

• JIRA_BASE_URL
• JIRA_PROJECT_KEY (or --project-key)
• TESTOPS_FORGE_ENDPOINT
• TESTOPS_FORGE_AUTH_TOKEN for the repo-owned authenticated webtrigger transport

One-time Forge transport setup:

cd apps/forge
forge deploy
forge install --upgrade --site automatify-com-au.atlassian.net --product Jira -e development --confirm-scopes --non-interactive
forge webtrigger create --functionKey cli-transport-webtrigger --site automatify-com-au.atlassian.net --product Jira -e development

Then open the Jira project page, go to Operations -> CLI access, create a CLI key, and copy the token immediately. It is shown once only.

Export the webtrigger URL and the one-time token for CLI use:

export TESTOPS_FORGE_ENDPOINT="<webtrigger-url>"
export TESTOPS_FORGE_AUTH_TOKEN="<one-time-token-from-forge-ui>"
export JIRA_BASE_URL="https://automatify-com-au.atlassian.net"
export JIRA_PROJECT_KEY="DEV"

Or store the non-secret values in .testops-cli.json and, on macOS, the token in Keychain:

automatify testops config set baseUrl https://automatify-com-au.atlassian.net
automatify testops config set projectKey DEV
automatify testops config set forgeEndpoint "<webtrigger-url>"
printf "%s" "$TESTOPS_FORGE_AUTH_TOKEN" | automatify testops config set forgeAuthToken --stdin

Windows, Linux, and CI should keep the token in environment variables:

export TESTOPS_FORGE_AUTH_TOKEN="<one-time-token-from-forge-ui>"

Transport notes:

• Bearer auth is required for operator/CI use. Forge stores only the token hash plus metadata.
• Tokens are project-scoped, shown once at creation time, and can be revoked from the Forge UI.
• Expired or revoked tokens are rejected by the transport.

Scenario automation callback example:

curl -sS "$TESTOPS_FORGE_ENDPOINT" \
  -H "authorization: Bearer $TESTOPS_FORGE_AUTH_TOKEN" \
  -H "content-type: application/json" \
  -d '{
    "contract":"updateScenarioAutomationStatus",
    "payload":{
      "context":{"projectKey":"DEV"},
      "scenarioId":"manual:checkout:guest-payment",
      "status":"running",
      "externalRunId":"gha-1024",
      "externalRunUrl":"https://github.com/Automatify-Pty-Ltd/automatify-jira-testops/actions/runs/1024",
      "responseStatus":202,
      "responseSnippet":"workflow is running"
    }
  }' | jq

You can also resolve the target by automationId or scenarioRef instead of scenarioId.

GitHub Actions callback example:

- name: Report scenario automation status to Forge
  if: ${{ always() }}
  env:
    TESTOPS_FORGE_ENDPOINT: ${{ secrets.TESTOPS_FORGE_ENDPOINT }}
    TESTOPS_FORGE_AUTH_TOKEN: ${{ secrets.TESTOPS_FORGE_AUTH_TOKEN }}
    TESTOPS_PROJECT_KEY: DEV
    TESTOPS_SCENARIO_REF: checkout.feature#3
    TESTOPS_STATUS: ${{ job.status == 'success' && 'passed' || job.status == 'cancelled' && 'cancelled' || 'failed' }}
  run: |
    curl -sS "$TESTOPS_FORGE_ENDPOINT" \
      -H "authorization: Bearer $TESTOPS_FORGE_AUTH_TOKEN" \
      -H "content-type: application/json" \
      -d "{
        \"contract\":\"updateScenarioAutomationStatus\",
        \"payload\":{
          \"context\":{\"projectKey\":\"$TESTOPS_PROJECT_KEY\"},
          \"scenarioRef\":\"$TESTOPS_SCENARIO_REF\",
          \"status\":\"$TESTOPS_STATUS\",
          \"externalRunId\":\"${GITHUB_RUN_ID}\",
          \"externalRunUrl\":\"https://github.com/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}\",
          \"responseStatus\":200,
          \"responseSnippet\":\"GitHub Actions callback from ${GITHUB_WORKFLOW}\"
        }
      }"

Azure DevOps callback example:

- bash: |
    set -euo pipefail
    if [ "$(Agent.JobStatus)" = "Succeeded" ]; then
      TESTOPS_STATUS="passed"
    elif [ "$(Agent.JobStatus)" = "Canceled" ]; then
      TESTOPS_STATUS="cancelled"
    else
      TESTOPS_STATUS="failed"
    fi

    curl -sS "$(TESTOPS_FORGE_ENDPOINT)" \
      -H "authorization: Bearer $(TESTOPS_FORGE_AUTH_TOKEN)" \
      -H "content-type: application/json" \
      -d "{
        \"contract\":\"updateScenarioAutomationStatus\",
        \"payload\":{
          \"context\":{\"projectKey\":\"DEV\"},
          \"scenarioRef\":\"checkout.feature#3\",
          \"status\":\"${TESTOPS_STATUS}\",
          \"externalRunId\":\"$(Build.BuildId)\",
          \"externalRunUrl\":\"$(System.CollectionUri)$(System.TeamProject)/_build/results?buildId=$(Build.BuildId)\",
          \"responseStatus\":200,
          \"responseSnippet\":\"Azure DevOps callback from $(Build.DefinitionName)\"
        }
      }"
  displayName: Report scenario automation status to Forge

Azure DevOps setup note:

• prefer pipeline variables for TESTOPS_FORGE_ENDPOINT
• prefer secret pipeline variables for TESTOPS_FORGE_AUTH_TOKEN
• do not keep those values in the Jira automation profile bodyTemplate

Core browse examples:

automatify testops bdd scenarios show --project-key DEV --id <SCENARIO_ID> --json
automatify testops bdd scenarios export --project-key DEV --id SC-4 --output-dir ./artifacts/scenario-export --json
automatify testops bdd scenarios import --project-key DEV --file ./artifacts/scenario-export/SC-4.feature --json
automatify testops bdd scenarios export --project-key DEV --all --output-dir ./artifacts/scenario-bulk --json
automatify testops bdd scenarios import --project-key DEV --input-dir ./artifacts/scenario-bulk --json
automatify testops bdd features export --project-key DEV --output-dir ./artifacts/features-export --zip --json
automatify testops runs show --project-key DEV --id <RUN_ID> --json
automatify testops config validate
automatify testops doctor check --json
automatify allure doctor --json

Advanced browse examples:

automatify testops cases list --project-key DEV
automatify testops cases show --project-key DEV --key TC-1 --json
automatify testops cases bdd list --project-key DEV --key TC-1 --json
automatify testops cases runs list --project-key DEV --key TC-1 --json
automatify testops suites list --project-key DEV --json
automatify testops suites cases list --project-key DEV --key TS-1

Recommended browse flow:

# Path A: scenario-first browse
automatify testops bdd scenarios show --project-key DEV --id <SCENARIO_ID> --json
automatify testops runs show --project-key DEV --id <RUN_ID> --json

# Path B: testcase-assisted browse
automatify testops cases list --project-key DEV
automatify testops cases show --project-key DEV --key TC-145 --json
automatify testops cases bdd list --project-key DEV --key TC-145 --json
automatify testops cases runs list --project-key DEV --key TC-145 --json
automatify testops bdd scenarios show --project-key DEV --id <SCENARIO_ID> --json
automatify testops runs show --project-key DEV --id <RUN_ID> --json

Feature-file round-trip:

# Export all Jira-managed BDD features as .feature files
automatify testops bdd features export --project-key DEV --output-dir ./artifacts/features-export --zip --json

# Export one scenario as SC-4.feature and import it back after editing
automatify testops bdd scenarios export --project-key DEV --id SC-4 --output-dir ./artifacts/scenario-export
automatify testops bdd scenarios import --project-key DEV --file ./artifacts/scenario-export/SC-4.feature

# Export every visible SC-* file and import the whole folder back later
automatify testops bdd scenarios export --project-key DEV --all --output-dir ./artifacts/scenario-bulk
automatify testops bdd scenarios import --project-key DEV --input-dir ./artifacts/scenario-bulk --dry-run
automatify testops bdd scenarios import --project-key DEV --input-dir ./artifacts/scenario-bulk

# Re-import one exported file later
automatify testops ingest feature --project-key DEV --file ./artifacts/features-export/001-checkout.feature

Feature export notes:

• bdd scenarios show --format feature and bdd scenarios show --feature render one scenario in feature-style text.
• bdd scenarios export writes a single-scenario SC-*.feature file with a reserved @testops-scenario-SC-* tag.
• Jira issue links are exported as normal Gherkin tags like @DEV-2; on import they are written back to linkedIssueKeys.
• current model has linked issues, not a separate persisted primary issue, so all exported issue-key tags are treated as equal links.
• bdd features export writes one .feature file per stored BDD feature plus manifest.json.
• with --zip, the CLI also writes ./artifacts/features-export.zip next to the export directory.
• pass --issue-key DEV-2 to export only features visible in that Jira issue context.
• getSummaryExport remains the reporting/export path for JSON/CSV summary snapshots and is separate from feature-file round-trip export.

Coverage and readiness report via the Forge transport:

curl -sS "$TESTOPS_FORGE_ENDPOINT" \
  -H "authorization: Bearer $TESTOPS_FORGE_AUTH_TOKEN" \
  -H "content-type: application/json" \
  -d '{"contract":"getCoverageReport","payload":{"context":{"projectKey":"DEV"}}}' | jq

Deterministic dry-run preview (human):

pnpm --filter @automatify-au/cli build
automatify testops auto \
  --dry-run \
  --root apps/cli/test-fixtures/auto/execution-layout

Deterministic dry-run preview (JSON for CI):

preview_json="$(automatify testops auto \
  --dry-run \
  --root apps/cli/test-fixtures/auto/execution-layout \
  --output json)"
echo "$preview_json"
node -e 'const p=JSON.parse(process.argv[1]); if(!p.preview || !p.preview.mapping){process.exit(2)}' "$preview_json"

Real execution mode (Forge upload path):

automatify testops auto \
  --root apps/cli/test-fixtures/auto/execution-layout \
  --project-key DEV \
  --force \
  --output json

Recommended CI sequence:

1. automatify testops config validate
2. automatify testops doctor check --json
3. automatify testops auto --dry-run --output json ...
4. automatify testops auto --force --output json ...

automatify testops auto MVP limits (explicit):

• detection scope is Playwright, pytest, cucumber/behave hints only.
• artifact scope is JUnit XML + Cucumber JSON + .feature files.
• mapping is deterministic MVP only (issue-key and supported exact-name matching).
• no AI-based matching, no backend-assisted matching, no storage surgery actions.

automatify testops auto pilot adoption checklist

1. Run deterministic preview:
   • automatify testops auto --dry-run --output json --root <repo-root>
2. Confirm preview is actionable:
   • framework mode is not unknown unless expected,
   • discovered artifacts count is non-zero for upload scenarios,
   • unmatched diagnostics are reviewed.
3. Verify readiness gate:
   • automatify testops doctor check --json
4. Run first real upload in controlled branch/commit:
   • automatify testops auto --force --output json --project-key <KEY> --root <repo-root>
5. Record command result and warnings in pilot notes.

Deterministic exit-code mapping

• 0 success
• 1 internal/runtime error
• 2 usage/argument errors
• 3 local validation errors
• 4 Forge service errors (ok: false payload)
• 5 Forge transport/connectivity errors

CI usage examples

Doctor preflight with JSON parsing:

set -euo pipefail
pnpm --filter @automatify-au/cli build
export TESTOPS_FORGE_ENDPOINT="${TESTOPS_FORGE_ENDPOINT:?missing}"
export JIRA_BASE_URL="${JIRA_BASE_URL:?missing}"
export JIRA_PROJECT_KEY="${JIRA_PROJECT_KEY:?missing}"

doctor_json="$(automatify testops doctor check --json)"
echo "$doctor_json"
node -e 'const d=JSON.parse(process.argv[1]); if(!d.status){process.exit(2)}' "$doctor_json"

Feature ingest from file:

pnpm --filter @automatify-au/cli build
automatify testops ingest feature \
  --file ./artifacts/checkout.feature \
  --project-key DEV

Bulk feature export to local .feature files:

pnpm --filter @automatify-au/cli build
automatify testops bdd features export \
  --project-key DEV \
  --output-dir ./artifacts/features-export \
  --zip \
  --json

Run upload from file with CI-friendly branching:

set +e
pnpm --filter @automatify-au/cli build
automatify testops run upload \
  --file ./artifacts/run.json \
  --project-key DEV \
  --json
code=$?
set -e

case "$code" in
  0) echo "run upload succeeded" ;;
  2|3) echo "usage/validation issue, fail fast" ; exit "$code" ;;
  4) echo "Forge service rejected payload" ; exit "$code" ;;
  5) echo "Forge connectivity issue, consider retry" ; exit "$code" ;;
  *) echo "unexpected failure" ; exit "$code" ;;
esac

Adoption checklist (MVP)

1. Build CLI:
   • pnpm --filter @automatify-au/cli build && pnpm --filter @automatify-au/cli bundle
2. Validate config:
   • automatify testops config validate
3. Run doctor preflight:
   • automatify testops doctor check --json
4. Ingest one sample feature:
   • automatify testops ingest feature --file ./artifacts/checkout.feature --project-key DEV
5. Upload one sample run:
   • automatify testops run upload --file ./artifacts/run.json --project-key DEV --json
6. (Optional) Verify sync:
   • TESTOPS_ENABLE_SYNC=1 automatify testops sync status

Runbook integration

• Pilot onboarding/operator triage snippets:
  • PILOT_ONBOARDING.md
• Release and rollback flow:
  • RELEASE_RUNBOOK.md