@automatify-au/cli

Minimal Forge-first CLI for Automatify Jira TestOps.

Current product stance:

• Jira UI is the primary day-to-day surface for most users.
• The CLI is the primary admin and automation surface for setup, ingestion, diagnostics, reporting, and CI.
• POC audience: operators, CI, internal support, and customer admins running controlled setup/import flows.
• Future customer-admin packaging is expected before any broader distribution.
• Any future all-user CLI requires a different auth/onboarding model than the current project-scoped admin key transport.

Command-group note:

• The groupings below describe intended usage and complexity only.
• They are not a security boundary in the current transport model.
• A valid project CLI token is not yet scoped per command group.

Architecture boundary (must hold)

• Thin client only.
• No separate backend.
• No duplicate domain/business logic in CLI.
• CLI commands must call shared Forge contracts (resolver/webtrigger paths) as they are introduced.
• Focused evidence commands may call Jira Cloud REST APIs directly when the feature is specifically Jira-native attachment/comment work.

MVP non-goals

• No separate backend.
• No duplicate domain/business logic in CLI.
• No replacement of Jira UI as a primary product surface.
• No backend-assisted/hybrid logic in this phase.

Install and run

From npm (published package)

npm install -g @automatify-au/cli
automatify --help
automatify testops --help

From repository (development)

pnpm install
pnpm --filter @automatify-au/cli build
pnpm --filter @automatify-au/cli bundle

This produces a single self-contained bundle at dist/automatify.cjs (≈125 KB) with @automatify/shared-types inlined. No runtime dependencies.

Run built CLI:

automatify testops --help            # if linked globally (npm link)
node apps/cli/dist/automatify.cjs --help  # direct node run

Dev run (for local iteration):

pnpm --filter @automatify-au/cli run cli:dev -- --help

Bundle and publish

cd apps/cli
npm test                      # optional but recommended local preflight
npm pack --dry-run            # inspect final tarball contents
npm publish --dry-run --access public  # simulate publish against npm
npm publish --access public   # publish to npm

Publish notes:

• prepack now rebuilds dist/automatify.js automatically, so npm pack and npm publish do not rely on a stale bundle.
• prepublishOnly runs the CLI test suite and then rebuilds the bundle before a real publish.
• Current package target is @automatify-au/cli; verify npm auth with npm whoami if needed before publishing.

Required configuration

Minimum:

• JIRA_BASE_URL
• At least one of JIRA_PROJECT_KEY or JIRA_ISSUE_KEY
• TESTOPS_FORGE_ENDPOINT for commands that call Forge contracts (doctor, ingest, run, sync); copy the installation-specific endpoint from Jira Automatify TestOps -> CLI
• TESTOPS_FORGE_AUTH_TOKEN; create a CLI key in Jira Automatify TestOps -> CLI and copy the one-time token when it is shown

Customer admins do not need the Forge developer CLI to obtain either value. The Jira CLI workspace resolves the webtrigger endpoint for the current app installation at runtime.

Local Jira auth helper on macOS/Windows:

printf "%s" "$JIRA_API_TOKEN" | automatify testops auth login \
  --base-url https://automatify-com-au.atlassian.net \
  --jira-email [email protected] \
  --token-stdin
automatify testops auth status

On Linux/CI, keep JIRA_API_TOKEN in the environment and run auth login without --token-stdin; the CLI stores only non-secret Jira settings. Use automatify testops auth logout to remove local Jira and Forge credentials while preserving non-secret URL/email settings. Environment variables are not modified by logout.

Forge transport auth remains an advanced setting. First open the Jira project, go to Automatify TestOps -> CLI, copy the displayed CLI endpoint, create a CLI key, and copy its one-time token. Then configure the CLI:

automatify testops config set projectKey DEV
automatify testops config set forgeEndpoint "<endpoint-copied-from-jira>"
printf "%s" "$TESTOPS_FORGE_AUTH_TOKEN" | automatify testops config set forgeAuthToken --stdin

Jira and Forge secrets use macOS Keychain on macOS and current-user Windows DPAPI on Windows. Linux and CI keep secrets in JIRA_API_TOKEN and TESTOPS_FORGE_AUTH_TOKEN. Existing plaintext local tokens remain readable for compatibility and can be migrated with automatify testops config migrate-secrets. See AUTH.md for the normal Jira auth workflow and WINDOWS.md for PowerShell setup and verification.

Optional:

• TESTOPS_FORGE_TIMEOUT_MS
• TESTOPS_FORGE_MAX_RETRIES
• TESTOPS_FORGE_RETRY_DELAY_MS
• TESTOPS_AUTH_MODE, JIRA_EMAIL, JIRA_API_TOKEN (when auth mode requires)

Command groups (MVP)

All TestOps commands are invoked as automatify testops <command>:

Core browse commands

• bdd
  • automatify testops bdd scenarios list
  • automatify testops bdd scenarios list --json
  • automatify testops bdd scenarios list --project-key <PROJECT_KEY> --issue-key <ISSUE_KEY>
  • automatify testops bdd scenarios show --id <SCENARIO_ID>
  • automatify testops bdd scenarios show --id <SCENARIO_ID> --format feature
  • automatify testops bdd scenarios show --id <SCENARIO_ID|SC-4> --feature
  • automatify testops bdd scenarios export --id <SCENARIO_ID|SC-4> --output-dir ./scenario-export
  • automatify testops bdd scenarios export --all --output-dir ./scenario-export
  • automatify testops bdd scenarios import --file ./scenario-export/SC-4.feature
  • automatify testops bdd scenarios import --file ./scenario-export/SC-4.feature --dry-run
  • automatify testops bdd scenarios import --input-dir ./scenario-export
  • automatify testops bdd scenarios import --input-dir ./scenario-export --dry-run
  • automatify testops bdd features export --output-dir ./features-export
  • automatify testops bdd features export --output-dir ./features-export --zip
  • automatify testops bdd features export --output-dir ./features-export --feature-id <FEATURE_ID>
• runs
  • automatify testops runs show --id <RUN_ID>
• auth
  • automatify testops auth login --base-url <JIRA_URL> --jira-email <EMAIL> --token-stdin
  • automatify testops auth status
  • automatify testops auth logout
• config
  • automatify testops config show
  • automatify testops config validate
• doctor
  • automatify testops doctor check
  • automatify testops doctor check --json
• allure
  • automatify allure doctor
  • automatify allure doctor --json
  • automatify allure doctor --issue-key ABC-123
  • automatify allure open ./allure-report.zip
  • automatify allure open ./allure-report.zip --no-open --port 8080

bdd scenarios list uses the configured project context or --project-key; --issue-key optionally narrows the Jira issue context. Human output is concise and selector-first:

BDD scenarios (1) for project DEV:
SC-14 | Guest payment | Feature: Checkout | Steps: 3 | Issues: DEV-2

With --json, the command emits one JSON object with stable action bdd-scenarios-list, projectKey, optional issueKey, count, and items.

Advanced browse commands

• cases
  • automatify testops cases list
  • automatify testops cases show --key <TEST_CASE_KEY>
  • automatify testops cases bdd list --key <TEST_CASE_KEY>
  • automatify testops cases runs list --key <TEST_CASE_KEY>
• suites
  • automatify testops suites list
  • automatify testops suites show --key <SUITE_KEY>
  • automatify testops suites cases list --key <SUITE_KEY>

Setup, ingestion, and CI commands

• setup
  • automatify testops setup plan ... > testops-setup-plan.json
  • automatify testops setup apply --plan testops-setup-plan.json --dry-run
  • automatify testops setup apply --plan testops-setup-plan.json --approve all --confirm ...
  • automatify testops setup doctor --plan testops-setup-plan.json ...
• ingest
  • automatify testops ingest feature --file <path> or --stdin
• run
  • automatify testops run upload --file <path> or --stdin
• allure
  • automatify testops allure upload ABC-123 ./allure-report.zip --dry-run
  • automatify testops allure upload ABC-123 ./allure-report.zip
  • automatify testops allure download ABC-123 --output-dir ./downloads
  • automatify testops allure open ./allure-report.zip
• auto
  • automatify testops auto --dry-run
  • automatify testops auto

Agent-friendly provider setup

automatify testops setup provides a GitHub-first, provider-bounded onboarding flow. The CLI remains a thin Forge client: Jira profile reads/writes/default selection use the existing listScenarioAutomationProfiles, upsertScenarioAutomationProfile, and setScenarioAutomationDefaultProfile contracts. Forge continues to own profile validation, tier limits, provider-secret storage, and default-profile rules.

1. Generate a deterministic plan

Run this from the repository that contains the approved workflow source:

automatify testops setup plan \
  --provider github-actions \
  --project-key DEV \
  --owner Automatify-Pty-Ltd \
  --repo customer-tests \
  --ref main \
  --workflow-id scenario-dispatch.yml \
  --workflow-source .github/workflows/scenario-dispatch.yml \
  --workflow-path .github/workflows/scenario-dispatch.yml \
  --profile-label "Customer GitHub Actions" \
  --set-default \
  > testops-setup-plan.json

The command itself writes nothing: it emits one versioned JSON plan to stdout, and the shell redirection above saves it. The plan contains the workflow source path and SHA-256 digest, not workflow contents or secret values. The source must declare workflow_dispatch and reference the selected callback secret names. Given the same explicit inputs and unchanged workflow source, the plan and planId are deterministic.

The GitHub plan covers:

• local workflow destination and approved source hash;
• repository, ref, workflow id, and Jira profile fields;
• required TESTOPS_FORGE_ENDPOINT and TESTOPS_FORGE_AUTH_TOKEN repository secret names;
• profile create/update, optional project-default selection, read-only smoke checks, and rollback guidance;
• explicit exclusion of account creation, credentials, commits, pull requests, and workflow runs.

2. Validate with a mutation-free dry-run

automatify testops setup apply \
  --plan testops-setup-plan.json \
  --repo-root /path/to/customer-tests \
  --dry-run

Dry-run previews all plan actions when no --approve is supplied. It may call read contracts to describe current state, but it does not write files, Jira profiles/defaults, GitHub secrets, commits, PRs, or workflow runs. Output is stable JSON.

3. Apply only explicitly approved scopes

Secrets must come from stdin or safe environment references. Secret values are rejected as ordinary command options and are not printed in JSON, logs, diagnostics, snapshots, or child-process arguments.

export TESTOPS_GITHUB_PROVIDER_TOKEN="<github-app-installation-or-fine-grained-token>"
export TESTOPS_GITHUB_ADMIN_TOKEN="<least-privilege-token-allowed-to-manage-actions-secrets>"
export TESTOPS_FORGE_ENDPOINT="<endpoint-copied-from-jira-cli-workspace>"
export TESTOPS_FORGE_AUTH_TOKEN="<one-time-project-cli-key>"

automatify testops setup apply \
  --plan testops-setup-plan.json \
  --repo-root /path/to/customer-tests \
  --approve all \
  --confirm \
  --secret-ref githubProviderToken=env:TESTOPS_GITHUB_PROVIDER_TOKEN \
  --secret-ref githubAdminToken=env:TESTOPS_GITHUB_ADMIN_TOKEN \
  --secret-ref callbackEndpoint=env:TESTOPS_FORGE_ENDPOINT \
  --secret-ref callbackAuthToken=env:TESTOPS_FORGE_AUTH_TOKEN

Available scopes are workflow-file, github-secrets, jira-profile, and project-default when the plan includes the default action. Real mutations require both --confirm and one or more --approve flags; use repeated flags for a partial apply. --approve all approves every action in the content-addressed plan.

The github-secrets scope uses the installed gh CLI, sends each repository secret through stdin, and passes the GitHub admin credential through GH_TOKEN in the child environment. Existing secret names are skipped because GitHub does not expose their values. Use --rotate-secrets to explicitly overwrite them. An existing matching Jira profile is also skipped; use --rotate-provider-token to explicitly replace its stored provider credential.

For JSON secret input instead of references, pipe one object with supported keys to --secrets-stdin:

node -e 'process.stdout.write(JSON.stringify({githubProviderToken:process.env.TESTOPS_GITHUB_PROVIDER_TOKEN,githubAdminToken:process.env.TESTOPS_GITHUB_ADMIN_TOKEN,callbackEndpoint:process.env.TESTOPS_FORGE_ENDPOINT,callbackAuthToken:process.env.TESTOPS_FORGE_AUTH_TOKEN}))' \
  | automatify testops setup apply \
      --plan testops-setup-plan.json \
      --repo-root /path/to/customer-tests \
      --approve all \
      --confirm \
      --secrets-stdin

Prefer a GitHub App installation token. When using fine-grained tokens, grant only the selected repository access plus the minimum Actions/Contents/Secrets permissions needed for the approved scopes. The command never creates or commits the workflow remotely; review and commit the local workflow through the repository's normal change-control process.

4. Run read-only provider doctor/smoke validation

automatify testops setup doctor \
  --plan testops-setup-plan.json \
  --repo-root /path/to/customer-tests \
  --secret-ref githubProviderToken=env:TESTOPS_GITHUB_PROVIDER_TOKEN \
  --secret-ref githubAdminToken=env:TESTOPS_GITHUB_ADMIN_TOKEN

Doctor verifies the local workflow hash, GitHub workflow metadata/content at the configured ref, required repository secret names, Jira profile fields/provider-secret metadata, and optional project default. It never dispatches a workflow. GitHub secret values cannot be read, so doctor verifies names only and reports that limitation. If the workflow has not been committed to the repository default branch, doctor reports what remains unverifiable without creating a commit or run.

Current authentication boundary:

• TESTOPS_FORGE_ENDPOINT is copied from the installation-specific endpoint displayed in Jira Automatify TestOps -> CLI.
• TESTOPS_FORGE_AUTH_TOKEN is a project-scoped, project-admin-created CLI key from the same Jira CLI workspace.
• The key authorizes the current Forge webtrigger transport for that project; command groups are not separate security scopes.
• Setup does not turn this into an all-user auth model. Customer-admin packaging and any future user-level authentication remain separate work.
• The same callback key may be stored as a GitHub Actions repository secret only under the explicit github-secrets apply scope.

Azure DevOps setup uses the same plan/apply/doctor safety boundary. Generate a plan with --provider azure-devops, mapping --organization, --azure-project, --pipeline-id, and --ref into the existing azureDevops Forge profile. Callback pipeline variables are accepted only through stdin or --secret-ref key=env:NAME; dry-run, explicit approval, idempotency, and redaction are identical to GitHub setup. Doctor inspects pipeline metadata and variable names only; it never queues a pipeline run.

Allure Evidence for Jira

Purpose:

• upload an already generated Allure HTML report ZIP to a Jira issue as evidence.
• download an uploaded Allure HTML report ZIP from a Jira issue for local inspection.
• parse widgets/summary.json from the ZIP and add a concise Jira comment with result counts.
• keep the flow Java-free and backend-free; this command does not generate Allure reports from raw allure-results.

Supported ZIP layouts:

• widgets/summary.json
• allure-report/widgets/summary.json
• any nested path ending in /widgets/summary.json

Credentials:

export JIRA_SITE="https://example.atlassian.net"
export JIRA_EMAIL="[email protected]"
export JIRA_API_TOKEN="<atlassian-api-token>"

JIRA_BASE_URL and existing .testops-cli.json baseUrl are also accepted for the site URL. CLI flags take precedence:

automatify testops allure upload ABC-123 ./allure-report.zip \
  --site https://example.atlassian.net \
  --email [email protected] \
  --api-token "$JIRA_API_TOKEN"

Dry run:

automatify testops allure upload ABC-123 ./allure-report.zip --dry-run

Upload without a Jira comment:

automatify testops allure upload ABC-123 ./allure-report.zip --no-comment

Normal output prints the Jira issue, attachment filename, attachment id, and attachment URL when Jira returns them.

Machine-readable output:

automatify testops allure upload ABC-123 ./allure-report.zip --dry-run --json

After a real upload, JSON output includes Jira attachment metadata when Jira returns them:

{
  "issueKey": "ABC-123",
  "uploaded": true,
  "commentAdded": true,
  "dryRun": false,
  "summary": {
    "total": 128,
    "passed": 120,
    "failed": 5,
    "broken": 1,
    "skipped": 2,
    "unknown": 0
  },
  "attachment": {
    "filename": "allure-report.zip",
    "id": "10001",
    "self": "https://example.atlassian.net/rest/api/3/attachment/10001",
    "content": "https://example.atlassian.net/rest/api/3/attachment/content/10001",
    "size": 4096
  }
}

Download the latest Allure report ZIP attachment from a Jira issue:

automatify testops allure download ABC-123 --output-dir ./downloads

Download selection rules:

• --attachment-id <id> downloads one exact Jira attachment.
• --filename <name> downloads the newest attachment with that filename.
• without either flag, the CLI chooses the newest allure-report*.zip attachment, then falls back to the newest .zip attachment.
• existing local files are not overwritten unless --force is passed.

Download dry run / JSON:

automatify testops allure download ABC-123 --dry-run --json

Allure preflight:

automatify allure doctor

automatify allure doctor checks:

• Jira site/email/API token resolution for direct Allure upload/download.
• optional Jira issue access only when --issue-key is passed.
• unzip availability for manual local ZIP inspection.
• Java availability as a warning-only optional check for future raw allure-results workflows.
• Allure CLI availability as a warning-only optional check.

Generated Allure HTML ZIP upload/download/open does not require Java or the Allure CLI.

Optional live Jira issue check:

automatify allure doctor --issue-key ABC-123

Open a generated Allure HTML report locally

automatify allure open ./allure-report.zip

automatify allure open <zipPath> and automatify testops allure open <zipPath> extract the report into a temporary directory, start a Java-free Node static HTTP server on an ephemeral port, and open the URL in your default browser. This is useful after downloading an Allure ZIP from Jira.

Options:

• --port <n> fixed port (default 0 for an OS-assigned free port).
• --host <h> bind host (default 127.0.0.1).
• --extract-to <dir> extract into a chosen directory; the directory is preserved on shutdown.
• --keep preserves the auto-created temp extract directory on shutdown.
• --no-open skips browser launch for CI/headless checks.
• --json, --dry-run, --verbose.

The server stays alive until Ctrl+C / SIGTERM. URL path traversal and malformed paths are rejected.

Optional gated maintenance commands

• sync
  • enable with TESTOPS_ENABLE_SYNC=1
  • automatify testops sync status
  • automatify testops sync reconcile --confirm RECONCILE

automatify testops auto (MVP)

Purpose:

• deterministic onboarding/import helper for local and CI workflows.
• thin orchestration only; domain logic stays in Forge contracts/services.

Supported commands:

• automatify testops auto --dry-run
• automatify testops auto

Required env/config for real upload mode:

• JIRA_BASE_URL
• JIRA_PROJECT_KEY (or --project-key)
• TESTOPS_FORGE_ENDPOINT
• TESTOPS_FORGE_AUTH_TOKEN for the repo-owned authenticated webtrigger transport

Customer CLI transport setup (no Forge developer CLI required):

1. Open the Jira project and go to Automatify TestOps -> CLI.
2. Copy the CLI endpoint displayed for the current installation.
3. Create a project CLI key and copy the token immediately; it is shown once only.
4. Configure those two values in the local shell, CI secret store, or CLI config.

Repository maintainers deploy/install the Forge app through the repository's controlled deployment workflow. That internal deployment process is separate from customer CLI onboarding.

Export the installation endpoint and one-time token for CLI use:

export TESTOPS_FORGE_ENDPOINT="<endpoint-copied-from-jira>"
export TESTOPS_FORGE_AUTH_TOKEN="<one-time-token-from-jira-cli-workspace>"
export JIRA_BASE_URL="https://automatify-com-au.atlassian.net"
export JIRA_PROJECT_KEY="DEV"

Or store the non-secret values in .testops-cli.json; on macOS the token is stored in Keychain, and on Windows it is protected with current-user DPAPI:

automatify testops config set baseUrl https://automatify-com-au.atlassian.net
automatify testops config set projectKey DEV
automatify testops config set forgeEndpoint "<endpoint-copied-from-jira>"
printf "%s" "$TESTOPS_FORGE_AUTH_TOKEN" | automatify testops config set forgeAuthToken --stdin

On Linux and CI, keep the token in environment variables. Windows users can use the same config set forgeAuthToken --stdin flow from PowerShell; see WINDOWS.md.

export TESTOPS_FORGE_AUTH_TOKEN="<one-time-token-from-jira-cli-workspace>"

Transport notes:

• Bearer auth is required for operator/CI use. Forge stores only the token hash plus metadata.
• Tokens are project-scoped, shown once at creation time, and can be revoked from the Jira CLI workspace.
• The endpoint is resolved by the Forge runtime for the current installation; customers do not configure a Forge-side endpoint variable.
• Expired or revoked tokens are rejected by the transport.

Scenario automation callback example:

curl -sS "$TESTOPS_FORGE_ENDPOINT" \
  -H "authorization: Bearer $TESTOPS_FORGE_AUTH_TOKEN" \
  -H "content-type: application/json" \
  -d '{
    "contract":"updateScenarioAutomationStatus",
    "payload":{
      "context":{"projectKey":"DEV"},
      "scenarioId":"manual:checkout:guest-payment",
      "status":"running",
      "externalRunId":"gha-1024",
      "externalRunUrl":"https://github.com/Automatify-Pty-Ltd/automatify-jira-testops/actions/runs/1024",
      "responseStatus":202,
      "responseSnippet":"workflow is running"
    }
  }' | jq

You can also resolve the target by automationId or scenarioRef instead of scenarioId.

GitHub Actions callback example:

- name: Report scenario automation status to Forge
  if: ${{ always() }}
  env:
    TESTOPS_FORGE_ENDPOINT: ${{ secrets.TESTOPS_FORGE_ENDPOINT }}
    TESTOPS_FORGE_AUTH_TOKEN: ${{ secrets.TESTOPS_FORGE_AUTH_TOKEN }}
    TESTOPS_PROJECT_KEY: DEV
    TESTOPS_SCENARIO_REF: checkout.feature#3
    TESTOPS_STATUS: ${{ job.status == 'success' && 'passed' || job.status == 'cancelled' && 'cancelled' || 'failed' }}
  run: |
    curl -sS "$TESTOPS_FORGE_ENDPOINT" \
      -H "authorization: Bearer $TESTOPS_FORGE_AUTH_TOKEN" \
      -H "content-type: application/json" \
      -d "{
        \"contract\":\"updateScenarioAutomationStatus\",
        \"payload\":{
          \"context\":{\"projectKey\":\"$TESTOPS_PROJECT_KEY\"},
          \"scenarioRef\":\"$TESTOPS_SCENARIO_REF\",
          \"status\":\"$TESTOPS_STATUS\",
          \"externalRunId\":\"${GITHUB_RUN_ID}\",
          \"externalRunUrl\":\"https://github.com/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}\",
          \"responseStatus\":200,
          \"responseSnippet\":\"GitHub Actions callback from ${GITHUB_WORKFLOW}\"
        }
      }"

Azure DevOps callback example:

- bash: |
    set -euo pipefail
    if [ "$(Agent.JobStatus)" = "Succeeded" ]; then
      TESTOPS_STATUS="passed"
    elif [ "$(Agent.JobStatus)" = "Canceled" ]; then
      TESTOPS_STATUS="cancelled"
    else
      TESTOPS_STATUS="failed"
    fi

    curl -sS "$(TESTOPS_FORGE_ENDPOINT)" \
      -H "authorization: Bearer $(TESTOPS_FORGE_AUTH_TOKEN)" \
      -H "content-type: application/json" \
      -d "{
        \"contract\":\"updateScenarioAutomationStatus\",
        \"payload\":{
          \"context\":{\"projectKey\":\"DEV\"},
          \"scenarioRef\":\"checkout.feature#3\",
          \"status\":\"${TESTOPS_STATUS}\",
          \"externalRunId\":\"$(Build.BuildId)\",
          \"externalRunUrl\":\"$(System.CollectionUri)$(System.TeamProject)/_build/results?buildId=$(Build.BuildId)\",
          \"responseStatus\":200,
          \"responseSnippet\":\"Azure DevOps callback from $(Build.DefinitionName)\"
        }
      }"
  displayName: Report scenario automation status to Forge

Azure DevOps setup note:

• prefer pipeline variables for TESTOPS_FORGE_ENDPOINT
• prefer secret pipeline variables for TESTOPS_FORGE_AUTH_TOKEN
• do not keep those values in the Jira automation profile bodyTemplate

Core browse examples:

automatify testops bdd scenarios list --project-key DEV --json
automatify testops bdd scenarios show --project-key DEV --id <SCENARIO_ID> --json
automatify testops bdd scenarios export --project-key DEV --id SC-4 --output-dir ./artifacts/scenario-export --json
automatify testops bdd scenarios import --project-key DEV --file ./artifacts/scenario-export/SC-4.feature --json
automatify testops bdd scenarios export --project-key DEV --all --output-dir ./artifacts/scenario-bulk --json
automatify testops bdd scenarios import --project-key DEV --input-dir ./artifacts/scenario-bulk --json
automatify testops bdd features export --project-key DEV --output-dir ./artifacts/features-export --zip --json
automatify testops runs show --project-key DEV --id <RUN_ID> --json
automatify testops config validate
automatify testops doctor check --json
automatify allure doctor --json

Advanced browse examples:

automatify testops cases list --project-key DEV
automatify testops cases show --project-key DEV --key TC-1 --json
automatify testops cases bdd list --project-key DEV --key TC-1 --json
automatify testops cases runs list --project-key DEV --key TC-1 --json
automatify testops suites list --project-key DEV --json
automatify testops suites cases list --project-key DEV --key TS-1

Recommended browse flow:

# Path A: scenario-first browse
automatify testops bdd scenarios list --project-key DEV --json
automatify testops bdd scenarios show --project-key DEV --id <SCENARIO_ID> --json
automatify testops runs show --project-key DEV --id <RUN_ID> --json

# Path B: testcase-assisted browse
automatify testops cases list --project-key DEV
automatify testops cases show --project-key DEV --key TC-145 --json
automatify testops cases bdd list --project-key DEV --key TC-145 --json
automatify testops cases runs list --project-key DEV --key TC-145 --json
automatify testops bdd scenarios show --project-key DEV --id <SCENARIO_ID> --json
automatify testops runs show --project-key DEV --id <RUN_ID> --json

Feature-file round-trip:

# Export all Jira-managed BDD features as .feature files
automatify testops bdd features export --project-key DEV --output-dir ./artifacts/features-export --zip --json

# Export one scenario as SC-4.feature and import it back after editing
automatify testops bdd scenarios export --project-key DEV --id SC-4 --output-dir ./artifacts/scenario-export
automatify testops bdd scenarios import --project-key DEV --file ./artifacts/scenario-export/SC-4.feature

# Export every visible SC-* file and import the whole folder back later
automatify testops bdd scenarios export --project-key DEV --all --output-dir ./artifacts/scenario-bulk
automatify testops bdd scenarios import --project-key DEV --input-dir ./artifacts/scenario-bulk --dry-run
automatify testops bdd scenarios import --project-key DEV --input-dir ./artifacts/scenario-bulk

# Re-import one exported file later
automatify testops ingest feature --project-key DEV --file ./artifacts/features-export/001-checkout.feature

Feature export notes:

• bdd scenarios show --format feature and bdd scenarios show --feature render one scenario in feature-style text.
• bdd scenarios export writes a single-scenario SC-*.feature file with a reserved @testops-scenario-SC-* tag.
• Jira issue links are exported as normal Gherkin tags like @DEV-2; on import they are written back to linkedIssueKeys.
• current model has linked issues, not a separate persisted primary issue, so all exported issue-key tags are treated as equal links.
• bdd features export writes one .feature file per stored BDD feature plus manifest.json.
• with --zip, the CLI also writes ./artifacts/features-export.zip next to the export directory.
• pass --issue-key DEV-2 to export only features visible in that Jira issue context.
• getSummaryExport remains the reporting/export path for JSON/CSV summary snapshots and is separate from feature-file round-trip export/import.

Coverage and readiness report via the Forge transport:

curl -sS "$TESTOPS_FORGE_ENDPOINT" \
  -H "authorization: Bearer $TESTOPS_FORGE_AUTH_TOKEN" \
  -H "content-type: application/json" \
  -d '{"contract":"getCoverageReport","payload":{"context":{"projectKey":"DEV"}}}' | jq

Deterministic dry-run preview (human):

pnpm --filter @automatify-au/cli build
automatify testops auto \
  --dry-run \
  --root apps/cli/test-fixtures/auto/execution-layout

Deterministic dry-run preview (JSON for CI):

preview_json="$(automatify testops auto \
  --dry-run \
  --root apps/cli/test-fixtures/auto/execution-layout \
  --output json)"
echo "$preview_json"
node -e 'const p=JSON.parse(process.argv[1]); if(!p.preview || !p.preview.mapping){process.exit(2)}' "$preview_json"

Real execution mode (Forge upload path):

automatify testops auto \
  --root apps/cli/test-fixtures/auto/execution-layout \
  --project-key DEV \
  --force \
  --output json

Recommended CI sequence:

1. automatify testops config validate
2. automatify testops doctor check --json
3. automatify testops auto --dry-run --output json ...
4. automatify testops auto --force --output json ...

automatify testops auto MVP limits (explicit):

• detection scope is Playwright, pytest, cucumber/behave hints only.
• artifact scope is JUnit XML + Cucumber JSON + .feature files.
• mapping is deterministic MVP only (issue-key and supported exact-name matching).
• no AI-based matching, no backend-assisted matching, no storage surgery actions.

automatify testops auto pilot adoption checklist

1. Run deterministic preview:
   • automatify testops auto --dry-run --output json --root <repo-root>
2. Confirm preview is actionable:
   • framework mode is not unknown unless expected,
   • discovered artifacts count is non-zero for upload scenarios,
   • unmatched diagnostics are reviewed.
3. Verify readiness gate:
   • automatify testops doctor check --json
4. Run first real upload in controlled branch/commit:
   • automatify testops auto --force --output json --project-key <KEY> --root <repo-root>
5. Record command result and warnings in pilot notes.

Deterministic exit-code mapping

• 0 success
• 1 internal/runtime error
• 2 usage/argument errors
• 3 local/plan validation errors, doctor failures, or post-apply smoke validation failures
• 4 Forge service errors (ok: false payload)
• 5 Forge transport/connectivity errors

CI usage examples

Doctor preflight with JSON parsing:

set -euo pipefail
pnpm --filter @automatify-au/cli build
export TESTOPS_FORGE_ENDPOINT="${TESTOPS_FORGE_ENDPOINT:?missing}"
export JIRA_BASE_URL="${JIRA_BASE_URL:?missing}"
export JIRA_PROJECT_KEY="${JIRA_PROJECT_KEY:?missing}"

doctor_json="$(automatify testops doctor check --json)"
echo "$doctor_json"
node -e 'const d=JSON.parse(process.argv[1]); if(!d.status){process.exit(2)}' "$doctor_json"

Feature ingest from file:

pnpm --filter @automatify-au/cli build
automatify testops ingest feature \
  --file ./artifacts/checkout.feature \
  --project-key DEV

Bulk feature export to local .feature files:

pnpm --filter @automatify-au/cli build
automatify testops bdd features export \
  --project-key DEV \
  --output-dir ./artifacts/features-export \
  --zip \
  --json

Run upload from file with CI-friendly branching:

set +e
pnpm --filter @automatify-au/cli build
automatify testops run upload \
  --file ./artifacts/run.json \
  --project-key DEV \
  --json
code=$?
set -e

case "$code" in
  0) echo "run upload succeeded" ;;
  2|3) echo "usage/validation issue, fail fast" ; exit "$code" ;;
  4) echo "Forge service rejected payload" ; exit "$code" ;;
  5) echo "Forge connectivity issue, consider retry" ; exit "$code" ;;
  *) echo "unexpected failure" ; exit "$code" ;;
esac

Adoption checklist (MVP)

1. Build CLI:
   • pnpm --filter @automatify-au/cli build && pnpm --filter @automatify-au/cli bundle
2. Validate config:
   • automatify testops config validate
3. Run doctor preflight:
   • automatify testops doctor check --json
4. Ingest one sample feature:
   • automatify testops ingest feature --file ./artifacts/checkout.feature --project-key DEV
5. Upload one sample run:
   • automatify testops run upload --file ./artifacts/run.json --project-key DEV --json
6. (Optional) Verify sync:
   • TESTOPS_ENABLE_SYNC=1 automatify testops sync status

Runbook integration

• Pilot onboarding/operator triage snippets:
  • PILOT_ONBOARDING.md
• Release and rollback flow:
  • RELEASE_RUNBOOK.md