@consiliency/pipeline-runtime

Runtime, harness, and workflow modules for the governed-pipeline engine. The package owns BAML boundaries, executor adapters, runtime ledger/evidence stores, governance and Portal projection bridges, the runtime-agent surface, and the phase-loop closeout ingest path.

BAML v1 runtime

The runtime uses @boundaryml/[email protected] on Node >=24.0.0 and npm >=10. Published packages load the current baml/precompiled.json bytecode once per process. An absent artifact, or a well-shaped, digest-valid artifact with stale source/version hashes, falls back to compiling the shipped baml/baml_src files. Custom sourceDir or registryPath options always compile source. An unreadable, malformed or corrupt artifact rejects initialization; a current blob rejected by the native bridge also fails permanently without source retry. The bridge builds a request for the external harness and parses its reply without calling a model. warmBamlRuntime() stays before the brownfield stage timers and at BAML-using CLI entry points, with no budget changes.

Normal install, default tests, agent:fast and agent:gate require no BAML CLI, generation, toolchain download or BAML network access. Ordinary npm dependency and native-package downloads remain allowed before isolated runtime execution. Default prepack is inert and removes a leftover artifact; distribution smoke tests scrub builder opt-ins. The Linux publish verification explicitly opts into the separate toolchain build. Bytecode, generated clients/SDKs and profiles are never committed.

Developers who want baml check can install the BAML v1 wrapper from the BoundaryML GitHub releases, then run baml toolchain pin 0.21.0 in baml/. baml/baml.toml pins the toolchain; it has no generator. Do not casually run baml generate or commit precompiled.json, baml_client/, baml_sdk/, or .baml/ output.

During either source or blob initialization, the module temporarily sets BAML_TELEMETRY=off if unset or exported empty. Explicit nonempty values are honored; unset, empty and explicit off have the default isolation guarantee. This is the 0.21.0 control that disables both cloud telemetry (including an environment API key or saved login) and local recordings under $HOME/.baml/btel/. All four values are restored immediately after initialization, including when initialization fails: unset by deletion, empty as "", and nonempty verbatim. The legacy BAML_PROFILE_DIR="", BAML_TELEMETRY_DISABLED=1, and DO_NOT_TRACK=1 window remains, but the 0.21.0 runtime does not read those switches (the BAML CLI still honors DO_NOT_TRACK); BAML_PROFILE_DIR is absent from its source.

Both backends are tested in fresh processes across unset, empty and off telemetry with no credential, a local key, and a dummy cloud key with an invalid API URL. Warm-up, companion self-check, build and parse leave isolated cwd, HOME, TMPDIR and BAML_HOME empty and restore the environment exactly. The invalid URL is a telemetry startup tripwire; it does not trace every network syscall. Linux verification of the exact packaged candidate uses capable strace -f across full goldens and credential tripwires to reject Internet attempts, including failed calls, and transient file writes. Cross-platform JS hooks additionally assert zero attempts but cannot see native Rust sockets or transient native writes. Explicit enabling values opt out of default isolation; low with the invalid cloud URL is a separate rejection/restoration diagnostic.

Blob initialization passes only the bytecode, without generated TOML, to avoid generated telemetry policy and its btel writes. This loses generated-metadata validation, exact generated-toolchain comparison, telemetry policy/credential binding and policy.verify_build. The pinned builder validates TOML before omitting it; JS checks both bridge/toolchain getters and source hashes, and the native envelope still checks integrity/format. These checks do not replace the omitted metadata validation.

Warm-up reads the registry through @consiliency/pipeline-schema/pipeline-definition, so a nested npm install uses its own schema package. Tests can pass { sourceDir, registryPath } to warmBamlRuntime() in a fresh process to exercise the self-check with altered fixtures. Options that differ after initialization begins throw; a failed initialization stays failed for that process. These overrides are explicit test options; environment variables cannot redirect production prompt source or registry inventory.

Maintainer bytecode release

1. Use Linux x64 with glibc, Node 24 and npm >=10, a writable executable disposable directory (GP_BAML_BUILD_TMPDIR), and a private verified cache (GP_BAML_TOOLCHAIN_CACHE_DIR). GP_BAML_TOOLCHAIN_ARCHIVE can supply the pinned 0.21.0 archive; otherwise the opt-in builder acquires it with bounded retries. Every archive/cache hit checks size and SHA-256, and the CLI member has its own pinned size/digest.
2. Set GP_BAML_PREPACK=1 only for npm pack --silent in a disposable package copy. Require exactly one filename on stdout; verify that tarball with node scripts/prepack-baml.mjs --verify-tarball <tgz>, then run installed-runtime, forced-blob goldens, startup and Linux native-isolation checks against that same path. Postpack removes the copy's artifact. Retain proof evidence outside scratch before cleaning disposable builds, archives, copies and isolation directories on success or failure. A future bridge bump must re-pin the archive/member digests and emitter checks and rerun the proof gate; T9 rejects pin skew.
3. Transfer the unchanged verified runtime tgz, sibling tgzs and raw golden tests/fixtures through a same-run artifact with SHA-256 hashes, bridge pin and commit identity. Linux/macOS/Windows Node 24 consumers install that identical runtime normally, resolve the exact platform native package, and verify install lifecycle, no BAML execution/release-asset access, no generated paths, goldens and isolation. Windows uses the artifact's golden bytes, never checkout fixtures.
4. Before any publication, recheck the manifest, commit, runtime digest and structure. Publish the downloaded runtime tgz first, then schema, adapter-sdk and cli from the verified workspace; those siblings have no build step. Do not rebuild/repack the runtime or fall back to workspace publication. Folder prepublishOnly remains a backstop; tarball publication bypasses it, so verified bytes are the guarantee.
5. After a partial workflow failure, re-run all jobs in the whole workflow, rather than only failed jobs. Artifact names embed github.run_attempt, so the new attempt must produce its own verified artifact; consumers never look up the latest run. A partial publication still requires the separate operator repair described below before retrying publication.

Consiliency/governed-pipeline#163's owner approved this procedure and the no-BAML-network scope on 2026-10-11. Publication is non-atomic: a runtime success followed by a sibling failure can leave partial versions or temporarily unavailable sibling dependencies. npm OIDC authorization for the runtime tarball remains unverified until an authorized tag run. Stop on the first failure; repair or deprecation is a separate operator action, with no automatic fallback, deprecation or issue creation.

Optional dependency: dotfiles phase-loop-runtime

The phase-loop runtime referenced by governed-pipeline's closeout ingest, status projection, and blocker normalization lives in dotfiles as a vendored Python package: ~/code/dotfiles/vendor/phase-loop-runtime/. governed-pipeline is consumption-only and does not require the package to be installed — all current behavior runs on the in-repo JSON parsing path.

If you want the schema-import shim to resolve from the installed package instead of the fallback constant, install it into a Python virtualenv and export the detection flag:

export DOTFILES_DIR=~/code/dotfiles
python3 -m venv ~/.venvs/phase-loop-runtime
source ~/.venvs/phase-loop-runtime/bin/activate
pip install -e "file://$DOTFILES_DIR/vendor/phase-loop-runtime"
export PHASE_LOOP_RUNTIME_AVAILABLE=1

phase-loop and codex-phase-loop console scripts will then be on PATH. With PHASE_LOOP_RUNTIME_AVAILABLE=1, the shim at src/workflow/phase-loop-schema-import.mjs reports source: 'phase-loop-runtime' and includes the package version. Without the flag (or when the binary probe fails), it returns the frozen PHASE_LOOP_SCHEMA_FALLBACK and existing JSON parsing paths run unchanged.

The CI smoke test at test/phase-loop-runtime-install.test.mjs exercises the install in a temporary virtualenv. It skips automatically when:

• SKIP_PHASE_LOOP_RUNTIME_INSTALL_TEST=1 is set
• The dotfiles vendored package is not present at $PHASE_LOOP_RUNTIME_PACKAGE (defaults to ~/code/dotfiles/vendor/phase-loop-runtime)
• python3, the venv module, or pip is unavailable

Opt-in v17 metadata consumers

Each consumer reads a specific dotfiles v17 phase-loop field and returns a normalized record. All consumers default to no-op; pass { enabled: true } per call to activate. Records are frozen.

| Module | Reads | Use case | | --- | --- | --- | | workflow/phase-loop-execution-policy-consumer | execution_policy block from closeout | Surface per-phase resolved executor pin in audit metadata | | workflow/phase-loop-auto-recovery-consumer | metadata.manual_repair.auto_recovery: true | Distinguish auto-recovered phases from operator-recovered phases | | workflow/phase-loop-selected-executor-consumer | LoopEvent.selected_executor | Per-executor analytics, token-spread reporting | | workflow/phase-loop-outage-class-consumer | blocker.blocker_class === 'unretryable_external_outage' | Reducer that rotates provider on outage but holds provider on auth failure |

Example:

import { consumeOutageClass } from '@consiliency/pipeline-runtime/workflow/phase-loop-outage-class-consumer';

const record = consumeOutageClass(closeoutPayload, { enabled: true });
if (record?.retry_strategy === 'rotate-provider') {
  // wire downstream reducer
}

Each consumer has a happy-path and absent-path test in test/phase-loop-*-consumer.test.mjs.

Tests

npm test -w @consiliency/pipeline-runtime                                # full suite
npm test -w @consiliency/pipeline-runtime -- runtime-agent               # runtime-agent slice
npm test -w @consiliency/pipeline-runtime -- phase-loop-schema-import    # GPADOPT shim
SKIP_PHASE_LOOP_RUNTIME_INSTALL_TEST=1 npm test -w @consiliency/pipeline-runtime  # skip pip install smoke