@ash-mallick/browserstack-sync

Sync Playwright, Cypress, Rest Assured / JUnit (@Test in *.java), and CLI tests (JSON manifest) to CSV and optionally to BrowserStack Test Management. Follows the BrowserStack TM Standard Guidelines for folder hierarchy, naming, mandatory fields, and tagging taxonomy. Push pass / fail / skip from Report Portal, JUnit XML, Playwright JSON, or a generic JSON file into a test run.

FREE AI-powered test step extraction using Ollama — runs 100% locally, no data sent to cloud.

By Ashutosh Mallick

Install

npm install @ash-mallick/browserstack-sync

Or run without installing: npx @ash-mallick/browserstack-sync --csv-only

Run

Run from your project root (where specs and .env live). Without --all or --spec, the CLI prompts you to pick specs (TTY only).

Scripts in package.json:

{
  "scripts": {
    "sync:e2e": "am-browserstack-sync",
    "sync:e2e-csv": "am-browserstack-sync --csv-only"
  }
}

CLI reference

Sync one spec with Module and Feature folders

BrowserStack folder path: Project → Module → Feature → Test cases (cases are created in the Feature folder).

# One spec → Ecosystem Catalog / General / <15 test cases>
npx am-browserstack-sync \
  --spec=platform-openshift-tests.1.spec \
  --module="Ecosystem Catalog" \
  --feature=General

The --spec value can be the file base name, with or without extension. Comma-separate for multiple specs:

npx am-browserstack-sync \
  --spec=login.spec,checkout.spec \
  --module=Authentication \
  --feature=Login

Partial name match also works (e.g. --spec=openshift matches platform-openshift-tests.1.spec).

Sync all specs with hierarchy and tags

npx am-browserstack-sync --all \
  --module="Ecosystem Catalog" \
  --feature=General \
  --project-code=ASK \
  --release=2.4.0 \
  --env=staging \
  --platform=web \
  --owner="Jane Doe" \
  --priority=P2 \
  --regression

CSV only (no BrowserStack API)

npx am-browserstack-sync --csv-only --spec=platform-openshift-tests.1.spec

Minimal tags (§05 taxonomy only)

npx am-browserstack-sync \
  --spec=platform-openshift-tests.1.spec \
  --module="Ecosystem Catalog" \
  --feature=General \
  --tags-minimal

Or set "tagsMinimal": true in .am-browserstack-sync.json.

AI options

# Default: Ollama when running (llama3.2)
npx am-browserstack-sync --spec=login.spec

# Regex-only steps (no Ollama)
npx am-browserstack-sync --spec=login.spec --no-ai

# Different model
npx am-browserstack-sync --spec=login.spec --model=codellama

Working directory

npx am-browserstack-sync --cwd=/path/to/your-app --spec=login.spec --module=Auth --feature=Login

Sync flags

| Flag | Description | |------|-------------| | --csv-only | Write CSV files only; skip BrowserStack sync | | --all | Sync every discovered spec; no interactive prompt | | --spec=name | Sync only listed specs (comma-separated; name substring match) | | --module=name | Module folder under the project (§01) | | --feature=name | Feature folder under Module; test cases go here | | --project-code=code | Tag project:code (e.g. project:ASK) | | --release=version | Tag release:version (e.g. release:2.4.0) | | --env=name | Tag env:name (e.g. env:staging) | | --platform=name | Tag platform:name (e.g. platform:web) | | --owner=name | Default owner on all cases (§03) | | --priority=P1 … P4 | Default priority (default: P3) | | --regression | Add regression tag to all synced cases | | --tags-minimal | Omit helper tags (spec/title slugs); keep §05 + sync tags | | --no-ai | Disable Ollama; use regex step extraction | | --model=name | Ollama model (default: llama3.2) | | --cwd=path | Project root (default: current directory) |

Folder behavior

| Situation | Result | |-----------|--------| | --module + --feature set | Cases go to Module / Feature | | Only folderHierarchy in config | Uses config array, e.g. ["Authentication","Login"] | | Neither set | Inferred from spec path (last two path segments) | | Spec under shared/ | Cases go to project SHARED folder | | test.skip / deprecated | Cases go to _Archive |

Config vs CLI: CLI --module / --feature override folderHierarchy in .am-browserstack-sync.json for that run.

push-results flags

Subcommand: npx am-browserstack-sync push-results [options]

| Flag | Description | |------|-------------| | --test-run=TR-12345 | Target test run ID (or BROWSERSTACK_TEST_RUN_ID) | | --no-latest-run | Do not auto-pick latest run; require --test-run | | --results-from=auto\|allure\|report-portal | Result source (default: auto) | | --allure-results=path | allure-results/ or allure-report/ directory | | --junit=path | JUnit XML file or directory | | --junit-hint=framework | Hint for JUnit parser when needed | | --playwright-report=path | Playwright JSON report file | | --results-json=path | Generic results JSON file | | --report-portal-link=url | Report Portal launch URL (pin launch) | | --cwd=path | Project root |

Example after a test run:

npx am-browserstack-sync push-results \
  --test-run=TR-12345 \
  --allure-results=allure-results/

Common recipes

| Goal | Command | |------|---------| | First sync, one spec, fixed folders | npx am-browserstack-sync --spec=my.spec --module=MyModule --feature=MyFeature | | CI: all specs + metadata | npx am-browserstack-sync --all --module=X --feature=Y --project-code=ASK --no-ai | | Regenerate CSVs only | npx am-browserstack-sync --csv-only --spec=my.spec | | Re-sync after code change | Same as first sync; unchanged cases are Skipped | | Push Allure to latest run | npx am-browserstack-sync push-results --allure-results=allure-results/ |

BrowserStack TM Standard Guidelines

This package enforces the BrowserStack Test Management Standard Guidelines out of the box.

§01 Folder & Suite Structure

BrowserStack TM uses a 4-level hierarchy: Project → Module → Feature → Test Cases (max depth: 2 folder levels under the project — Module and Feature).

Test cases are stored directly in the Feature folder (not in a spec-name subfolder).

Use --module and --feature on the CLI (or folderHierarchy in config) to set the hierarchy. See CLI reference for full examples.

# Project → Authentication → Login → <test cases>
npx am-browserstack-sync --spec=login.spec \
  --module=Authentication \
  --feature=Login

When --module / --feature are omitted, Module and Feature are inferred from the spec file path (deeper paths are flattened to the last two segments).

Special reserved folders:

• _Archive — deprecated or removed cases are moved here (never deleted), tagged deprecated
• SHARED — reusable cases under a shared/ path are stored here to avoid duplication

You can also set the hierarchy in your config file:

{
  "folderHierarchy": ["Authentication", "Login"]
}

§02 Test Case Naming

• Titles are automatically normalized to sentence case (e.g. Verify user is able to log in successfully)
• IDs are zero-padded integers: TC-001, TC-002, ..., TC-100
• No spaces, slashes, or special chars in IDs — hyphens only

§03 Mandatory Fields

Every synced test case includes all mandatory fields:

| Field | Source | |-------|--------| | title | Extracted from spec, sentence-cased | | description | Auto-generated from framework + spec info | | preconditions | From case metadata or config (default: empty) | | steps | AI (Ollama) or regex extraction | | expected_results | Derived from steps | | owner | --owner flag / BROWSERSTACK_DEFAULT_OWNER env / config | | state | Active (default) | | priority | --priority flag / config defaultPriority (default: P3) | | case_type | Functional (default) | | linked_ticket | From case metadata (Jira ID recommended) | | test_data_ref | From case metadata | | automation_status | automated (default) |

Set defaults via config file, env vars, or CLI flags (see Config section below).

§05 Tagging Taxonomy

Tags are automatically generated following the standard taxonomy:

| Tag | How to set | |-----|-----------| | project:ASK | --project-code=ASK | | release:2.4.0 | --release=2.4.0 | | automation-ready / manual-only | Based on automation_status | | regression | --regression flag or spec name contains "regression" | | smoke | Automatic when --priority=P1 | | deprecated | Automatic for test.skip or removed cases archived to _Archive | | shared | Automatic for specs under a shared/ directory | | env:staging | --env=staging | | platform:web | --platform=web | | framework tag | Automatic: playwright, cypress, rest-assured, cli | | scope tag | Automatic: e2e or api |

By default the sync also adds internal helper tags (spec slug, title slug) and TC-xxx for re-sync matching. To upload only §05 taxonomy tags plus TC-xxx:

npx am-browserstack-sync --all --tags-minimal

Or in config: "tagsMinimal": true

Important: TC-001, TC-002, … are per spec file, not globally unique in a BrowserStack project. The sync always adds a spec slug tag on upload (e.g. platform_openshift_tests_1) and matches existing cases by spec slug + TC-xxx, so one spec’s TC-006 never updates another spec’s TC-006. Title-only matching also requires the same spec scope.

Full example with all standard tags:

npx am-browserstack-sync --all \
  --project-code=ASK \
  --release=2.4.0 \
  --module=Authentication \
  --feature=Login \
  --env=staging \
  --platform=web \
  --owner="Jane Doe" \
  --priority=P2 \
  --regression

Config (optional)

Defaults: e2e dir playwright/e2e, CSV dir playwright/e2e-csv.

Override via .am-browserstack-sync.json in project root:

{
  "e2eDir": "playwright/e2e",
  "csvOutputDir": "playwright/e2e-csv",
  "projectCode": "ASK",
  "releaseVersion": "2.4.0",
  "environment": "staging",
  "platform": "web",
  "owner": "Jane Doe",
  "defaultPriority": "P3",
  "folderHierarchy": ["Authentication", "Login"]
}

Or package.json field "amBrowserstackSync": { ... }.

Environment variable overrides:

| Env var | Config field | |---------|-------------| | PLAYWRIGHT_BROWSERSTACK_E2E_DIR | e2eDir | | PLAYWRIGHT_BROWSERSTACK_CSV_DIR | csvOutputDir | | BROWSERSTACK_PROJECT_CODE | projectCode | | BROWSERSTACK_RELEASE_VERSION | releaseVersion | | BROWSERSTACK_ENVIRONMENT | environment | | BROWSERSTACK_PLATFORM | platform | | BROWSERSTACK_DEFAULT_OWNER | owner | | BROWSERSTACK_DEFAULT_PRIORITY | defaultPriority |

Unified multi-framework layout (testSources)

When you need more than one kind of test in the same project, add a testSources array:

{
  "csvOutputDir": "test-output/csv",
  "projectCode": "ASK",
  "testSources": [
    { "framework": "playwright", "dirs": ["playwright/e2e"] },
    { "framework": "cypress", "dirs": ["cypress/e2e"] },
    { "framework": "rest-assured", "dirs": ["src/test/java/com/myapp/api"] },
    { "framework": "cli", "manifest": "cli-tests.json" }
  ]
}

• playwright / cypress — scans recursively for *.spec.*, *.test.*, or *.cy.*.
• rest-assured (alias: junit / java) — scans *.java with @Test; rows named ClassName.methodName.
• cli — manifest is a JSON file: { "cases": [ { "id": "TC-001", "title": "Smoke script" } ] } (IDs optional).

If testSources is omitted, a single e2eDir (non-recursive) is used.

AI-Powered Step Analysis (FREE with Ollama)

The tool uses Ollama to analyze your test code and generate human-readable test steps following §04 step-writing rules (one action per step, observable expected results). Ollama runs 100% locally — no data sent to cloud, completely free, no API key needed.

Example transformation:

test('should log in successfully', async ({ page }) => {
  await page.goto('/login');
  await page.getByLabel(/email/i).fill('[email protected]');
  await page.getByLabel(/password/i).fill('validpassword');
  await page.getByRole('button', { name: /sign in/i }).click();
  await expect(page).toHaveURL(/\/dashboard/);
});

Generated steps:

| Step | Expected Result | |------|-----------------| | Navigate to /login page | Login page loads successfully | | Enter '[email protected]' in the Email field | Email is entered | | Enter 'validpassword' in the Password field | Password is masked and entered | | Click the blue 'Sign In' button at the bottom of the form | Form is submitted | | Verify URL | URL matches /dashboard |

Setup Ollama

# 1. Install from https://ollama.ai, then:
ollama pull llama3.2
ollama serve

# 2. AI analysis is automatic when Ollama is running:
npx am-browserstack-sync --csv-only

Options:

• --no-ai — Use code-derived steps only (parses expect(), getByTestId, getByRole, etc. from the spec — best for UI validation tests)
• --model=llama3.2 — Use a different Ollama model
• OLLAMA_MODEL=llama3.2 — Set default model via env
• OLLAMA_HOST=http://localhost:11434 — Custom Ollama host

Accurate steps for catalog / breadcrumb / band tests: If Ollama invents login or form steps that are not in your spec, use --no-ai or upgrade to the latest package — AI output is grounded in the test code and falls back to assertion-based extraction when the model hallucinates.

| Model | Size | Best for | |-------|------|----------| | llama3.2 | 2GB | General purpose, fast (default) | | codellama | 4GB | Better code understanding | | llama3.2:1b | 1GB | Fastest, lower quality | | mistral | 4GB | Good balance |

Push results to a BrowserStack test run

After you have test cases and a test run (TR-...) in Test Management, you can record execution status from any of the supported report sources.

Allure Report (raw results or compiled)

The package supports both Allure output formats:

| Format | Path | When to use | |--------|------|-------------| | Raw results | allure-results/ | Generated during test run (allure.results.directory) — most accurate | | Compiled report | allure-report/ | Generated by allure generate — full HTML report |

# From raw allure-results/ directory (recommended)
npx am-browserstack-sync push-results \
  --test-run=TR-12345 \
  --allure-results=allure-results/

# From compiled allure-report/ directory
npx am-browserstack-sync push-results \
  --test-run=TR-12345 \
  --allure-results=allure-report/

# Use --results-from=allure to auto-detect allure-results/ or allure-report/ in project root
npx am-browserstack-sync push-results \
  --results-from=allure \
  --test-run=TR-12345

# Use latest run (auto-pick from project)
npx am-browserstack-sync push-results \
  --allure-results=allure-results/

Allure status mapping:

| Allure status | BrowserStack status | |---------------|-------------------| | passed | passed | | failed | failed | | broken | failed (unexpected exception) | | skipped | skipped | | unknown | skipped |

Auto-detection in CI: set ALLURE_RESULTS_DIR or ALLURE_REPORT_DIR env vars and use the default --results-from=auto — Allure is checked before JUnit/Playwright fallbacks.

TC-xxx tag matching: Allure labels named testCaseId, browserstackTestCaseId, or AS_ID are mapped directly to BrowserStack test case identifiers, enabling exact matching without title comparison.

JUnit XML

npx am-browserstack-sync push-results --test-run=TR-12345 --junit=target/surefire-reports

Playwright JSON

npx am-browserstack-sync push-results --playwright-report=test-results.json

Or use the latest run in the project (auto-picked by created_at):

npx am-browserstack-sync push-results --playwright-report=test-results.json

Report Portal

export REPORT_PORTAL_URL=https://your-rp.example.com
export REPORT_PORTAL_PROJECT=your_project
export REPORT_PORTAL_API_KEY=your_uuid_token
# optional: pin a launch from the UI
export REPORT_PORTAL_LAUNCH_LINK='https://your-rp.example.com/ui/#your_project/launches/all/<launch-uuid>'

npx am-browserstack-sync push-results --results-from=report-portal --test-run=TR-12345

Auto fallback order (--results-from=auto, default)

1. Report Portal (if REPORT_PORTAL_* env vars are set)
2. Allure — ALLURE_RESULTS_DIR env → allure-results/ in cwd → ALLURE_REPORT_DIR env → allure-report/ in cwd
3. JUnit — JUNIT_RESULTS_PATH / CYPRESS_JUNIT_RESULTS env
4. Playwright JSON — PLAYWRIGHT_JSON_REPORT env → test-results.json in cwd
5. Generic JSON — TEST_RESULTS_JSON env

Generic JSON

{
  "results": [
    { "title": "Login flow", "fullName": "e2e > auth > Login flow", "status": "passed" },
    { "title": "TC-003 -- checkout", "status": "failed", "browserstackTestCaseId": "TC-99999" }
  ]
}

npx am-browserstack-sync push-results --results-json=./results.json --test-run=TR-12345

Matching order: browserstackTestCaseId -> TC-xxx tag in title -> title / fullName substring.

BrowserStack sync setup

1. Create .env in project root (do not commit):

   BROWSERSTACK_USERNAME=your_username
   BROWSERSTACK_ACCESS_KEY=your_access_key
   BROWSERSTACK_PROJECT_ID=PR-XXXX
   # or: BROWSERSTACK_API_TOKEN=your_token

2. Get credentials from Test Management -> API keys. Project ID is in the project URL (PR-1234).

3. Run sync (example: one spec with Module / Feature):

   npx am-browserstack-sync \
     --spec=login.spec \
     --module=Authentication \
     --feature=Login \
     --project-code=ASK \
     --release=2.4.0 \
     --owner="Your Name"

   Sync everything:

   npx am-browserstack-sync --all \
     --project-code=ASK \
     --module=Authentication \
     --feature=Login \
     --release=2.4.0

What it does

• Discovers Playwright / Cypress specs, Java @Test methods, and CLI manifest cases.
• Extracts test titles; assigns zero-padded IDs (TC-001, TC-002, ...) and normalizes titles to sentence case (§02).
• Analyzes test code with Ollama AI or regex to generate human-readable steps following §04 step-writing rules.
• Enriches each case with all §03 mandatory fields: priority, owner, preconditions, state, case_type, linked_ticket, test_data_ref.
• Builds §05 tag taxonomy: project:, release:, automation-ready/manual-only, regression, smoke, env:, platform:.
• Creates §01 folder hierarchy: Module / Feature (test cases live in the Feature folder).
• Writes one CSV per spec with all mandatory columns.
• Syncs to BrowserStack: creates/updates test cases; skips unchanged ones.

Author: Ashutosh Mallick