@fabricorg/databricks-bdd

Gherkin (cucumber-js) step library for testing Databricks artifacts (ADR-0006). The same .feature file runs hermetically on DuckDB under DBX_TEST_PROFILE=local and against a real SQL warehouse under DBX_TEST_PROFILE=live (gated by DBX_TEST_LIVE=1). Live-only scenarios are tagged (@live, @jobs, @dlt, @lakebase, @slow) and auto-skip locally. Runtime predicates such as @requires.cloud=azure and Behave-compatible @use.with_cloud=azure tags skip scenarios when configured capabilities do not match.

Scenario: Seed a table and assert its contents
  Given a table "conversions" with:
    | subject_id | event_name | value | at                   |
    | u1         | purchase   | 49.0  | 2026-07-01T12:00:00Z |
  Then table "conversions" has 1 rows
  And table "conversions" contains rows matching:
    | subject_id | value |
    | u1         | 49.0  |

Setup

cucumber.mjs in your package:

export default {
  paths: ['features/**/*.feature'],
  import: ['features/support/steps.ts'], // your steps; import the library from there
  format: [
    'progress',
    ['junit', 'reports/bdd-junit.xml'],
    ['html', 'reports/bdd.html'],
    ['rerun', 'reports/rerun.txt'],
    ['@fabricorg/databricks-bdd/evidence-formatter', 'reports/evidence.json'],
  ],
};

features/support/steps.ts:

import '@fabricorg/databricks-bdd/steps'; // registers the world, hooks, and step library
// ... your own domain steps, typed against DatabricksWorld

Run with the tsx loader: NODE_OPTIONS="--import tsx" cucumber-js. Or embed in vitest via runFeatures({ cwd }) from the package root export.

Local developer profiles:

pnpm test:features:pretty # colored, source-annotated output
pnpm test:features:wip    # @wip only, fail fast, summary output

Step library

Given: a Databricks workspace · catalog {string} and schema {string} · a table {string} with: (types inferred from cells) · a table {string} with schema {string} and rows: · an empty table {string} with schema {string} · a table {string} with schema {string} from fixture {string} (NDJSON). Live setup also includes a Lakebase database, scenario-scoped Volume fixture uploads, and secondary service-principal grant contexts with automatic cleanup.

When: I run the SQL: (docstring; errors are captured, not thrown) · I query table {string} · I run job {string} (live) · I start a full refresh of pipeline {string} (isolated live pipeline only) · I submit notebook {string} · I run dbt build for {string}.

Lakeflow Jobs orchestration additionally provides I submit the Databricks job orchestration: (JSON docstring), I cancel the current Databricks job run, and I repair failed tasks in the current Databricks job run.

Then: table {string} has {int} rows · table {string} contains rows matching: · table {string} contains exactly: · table {string} matches golden {string} ordered by {string} · the result has {int} rows · the result contains rows matching: · the result matches golden {string} · the statement fails with {string} · the statement fails with permission denied · no rows were rescued in table {string} · row count of {string} is within {int}% of {int} · the job run reaches {runState} · the pipeline update completes within {duration} · schema-contract assertions.

Task-graph assertions include task {string} reaches {runState} and the job run has {int} tasks. They reuse the same scenario World, failure attachments, redaction, and Quality evidence pipeline as SQL and BDD results.

The World supplies Behave-style scenario cleanup, retry-safe run/feature fixtures with reverse-order AfterAll cleanup, typed userdata, run-level live preflight, redacted failure attachments, per-step process-output capture, custom parameter types, layered run/feature/scenario state, typed action composition, ?/*/+ cardinality parsing, active capability tags, Scenario Outlines, serial live execution, and parallel local execution. Evidence JSON schema v2 includes structured SQL, statement ID, and captured-output failure records. Capture defaults to 32 KiB per step; use DBX_TEST_CAPTURE_OUTPUT=passthrough to show output while capturing or off to disable it. Generate the complete catalog, including unused definitions, with pnpm steps:catalog.

const seeded = await this.featureFixture('reference-data', async () => ({
  value: await seedReferenceData(),
  cleanup: deleteReferenceData,
}))

Feature fixtures are keyed by feature URI and cleaned at run end because cucumber-js has no safe after_feature event when filters, retries, and parallel workers are involved. Use this.addCleanup() for scenario scope and sharedFixtures.runFixture() for run scope.

TypeScript parity with Behave

The Python APIs map to typed TypeScript outcomes:

import {
  defineAction,
  parseCardinalityField,
  type DatabricksWorld,
} from '@fabricorg/databricks-bdd'

const selectColumns = defineAction<[string], string[]>((world, raw) => {
  const columns = parseCardinalityField(raw, '+', String)
  world.setState('scenario', 'columns', columns)
  return columns
})

// Inside a step:
await this.runAction(selectColumns, 'subject_id, event_name')
this.setState('feature', 'catalog', 'fx_test')
const catalog = this.requireState<string>('catalog')

This supplies the behavior of context.execute_steps(), hierarchical user context, active tags, and cfparse cardinality without a Python runtime. Nested textual steps are intentionally represented as ordinary typed actions, which remain unit-testable and preserve TypeScript stack traces.

Capability tags read DBX_TEST_CLOUD, DBX_TEST_COMPUTE_MODE, DBX_TEST_STAGE, and DBX_TEST_CAPABILITY_*. Supported predicates are @requires.<key>[=<value>], @excludes.<key>[=<value>], @use.with_<key>=<value>, and @not.with_<key>=<value>.

Keep feature text engine-neutral. The shipped framework is TypeScript end to end; Python adapters are optional ecosystem bridges and are not required or currently included.

Product-domain example (aggregates, variants, SRM guardrail): apps/api/features/ in this repo. Docs: apps/docs/content/docs/testing/bdd.mdx.