@cipherstash/protect-ffi
v0.32.0
Published
Native FFI bindings to the CipherStash Client SDK — powers @cipherstash/stack
Keywords
Readme
CipherStash Client FFI (@cipherstash/protect-ffi)
[!IMPORTANT] If you are looking to implement this package into your application please use the official
@cipherstash/stackpackage.
This project provides the JS bindings for the CipherStash Client Rust SDK and is bootstrapped by create-neon.
Building
Building requires a supported version of Node and Rust.
To run the build, run:
$ npm run buildThis command uses the @neon-rs/cli utility to assemble the binary Node addon from the output of cargo.
Local setup
Authenticate the stash CLI to set up your local environment. It runs via npx — no separate install needed:
npx stash auth loginYou will be prompted to sign in or create an account.
Exploring
After building protect-ffi, you can explore its exports at the Node console.
Credentials must be available for the call to newClient() to succeed — resolved from the CipherStash profile store, from environment variables, or from an explicit authStrategy passed to newClient(). For local development the simplest option is npx stash auth login, which populates the profile store. Alternatively, set CS_WORKSPACE_CRN, CS_ACCESS_KEY, and the CS_CLIENT_ID/CS_CLIENT_KEY keypair in your environment.
$ npm i
$ npm run build
$ node
> const addon = require(".");
> const client = await addon.newClient({ encryptConfig: {v: 1, tables: {users: {email: {indexes: {ore: {}, match: {}, unique: {}}}}}} });
> const ciphertext = await addon.encrypt(client, { plaintext: "plaintext", column: "email", table: "users" });
> const plaintext = await addon.decrypt(client, { ciphertext });
> console.log({ciphertext, plaintext});EQL version selection
newClient accepts an eqlVersion option selecting the wire format that
encrypt / encryptBulk / encryptQuery emit:
// EQL v2 (default) — the `eql_v2_encrypted` payload format
const v2 = await addon.newClient({ encryptConfig })
// EQL v3 — payloads for the per-capability `eql_v3` domains
const v3 = await addon.newClient({ encryptConfig, eqlVersion: 3 })With eqlVersion: 3, each column's payload targets the eql_v3 domain
derived from its cast_as and indexes:
Every public-schema column domain carries an eql_v3_ prefix (so the SQL
column type is public.eql_v3_text_eq); the term-only query twins do not
(eql_v3.query_text_eq), because the eql_v3 schema already versions them.
| cast_as | family | indexes | domain |
|-----------|--------|---------|--------|
| text | text | unique + ore + match | eql_v3_text_search_ore |
| text | text | unique + ope + match | eql_v3_text_search |
| text | text | unique + ore | eql_v3_text_ord_ore |
| text | text | match | eql_v3_text_match |
| text | text | unique | eql_v3_text_eq |
| int / small_int / bigint | integer / smallint / bigint | ore (with or without unique) | eql_v3_<family>_ord_ore |
| int / small_int / bigint | integer / smallint / bigint | unique | eql_v3_<family>_eq |
| number / decimal / date / timestamp | double / numeric / date / timestamp | as above | as above |
| any scalar | — | none | storage-only domain (eql_v3_text, eql_v3_integer, …) |
| boolean | boolean | none only | eql_v3_boolean (storage-only — any index errors) |
| json | json | ste_vec (required, compat mode) | eql_v3_json_search |
Notes:
- The richest matching domain wins, and it must cover every configured
capability — a combination that would silently drop a term errors instead
(e.g.
unique+matchorore+matchon text: no single domain carries those term sets, so add the missing index to reach a search domain or split the capabilities across columns. Scalar-only configurations may useeqlVersion: 2; configurations containingste_vecrequire v3. - The two text search domains differ only in the ordering term they carry:
eql_v3_text_search_orecarriesob(ORE),eql_v3_text_searchcarriesop(OPE). Configuring bothoreandopeselects the ORE one. - Exception: dropping a term is fine when the capability survives.
Non-text ordering domains carry only
ob, sounique+oreon a numeric column dropshm— equality still works via the ORE operators. - Ordered text requires a
uniqueindex (eql_v3_text_ord_ore/eql_v3_text_ord_opecarryhm+ob/op);ore-only text errors. - A
jsoncolumn'sste_vecindex must use thecompatmode (thecipherstash-clientdefault since 0.40.0). v3 orders SteVec entries by the CLLW-OPEopterm; astandard-mode index emits CLLW-OREoc, which has no mechanical conversion, so such a column errors at config time. - Configurations containing
ste_vecdefault to EQL v3. ExpliciteqlVersion: 2is rejected because client 0.42 cannot represent the new selector-bound SteVec envelope in the v2 wire format. decryptaccepts both formats regardless ofeqlVersion, so v2 and v3 data can coexist during a migration.- v3 query encryption returns index-terms-only operands: scalar queries
produce the column domain's query twin (
{v, i, <terms>}, nocciphertext) — bind withcol = $1::jsonb::eql_v3.query_<name>(or the ordering operators, or@@for match). The operand always carries ALL the column domain's terms, whicheverindexTypewas queried. JSON containment queries produce theeql_v3.query_jsonneedle, andste_vec_selectorqueries return the bare selector hash (a string) for the->/->>operators. Exact JSON equality at a path usesste_vec_value_selectorwith{path, value}and returns a one-entry selector-only containment needle suitable for the GIN-backed@>operator. ope-indexed columns map to<family>_ord_opeand carry theop(CLLW-OPE) term (emitted since cipherstash-client 0.38.1).
[!NOTE] Breaking TypeScript change:
encrypt/encryptBulknow returnEncryptedPayload(Encrypted | EncryptedV3) instead ofEncrypted. Runtime output is unchanged for v2 clients, but code that accessed.kor assigned the result toEncryptedmust narrow first. v3 scalars carry nok, so guard its presence before reading it:'k' in payload && payload.k === 'ct'(a v2 scalar), or checkpayload.v === 3to detect the v3 members. (A barepayload.k === 'ct'does not compile against the union.)
BigInt plaintexts
Encrypted cast_as: 'bigint' columns store signed 64-bit integers
(PostgreSQL bigint). encrypt / encryptBulk / encryptQuery /
encryptQueryBulk accept the plaintext as either a JS number or a JS
bigint:
bigintinputs are exact and bounds-checked against the full i64 range: -9223372036854775808 to 9223372036854775807 (-2^63 to 2^63 - 1). Values outside that range throw aRangeErrornaming the bounds and the offending direction. It carries nocode— this is a boundary check in JS, not one of the Rust errors described under Errors. Search index terms (hm,ob,op) are derived from the same value, so the boundary check covers index-term generation too.numberinputs keep the existing exact-integer guard: fractional, non-finite, or out-of-range values error instead of being silently truncated.- A
bigintis only accepted as the top-level plaintext of a scalar column. JSON has no bigint, so bigints nested insidecast_as: 'json'documents (or JSON containment query terms) are rejected with aTypeErroron both Neon and wasm. More generally, plaintexts followJSON.stringifysemantics on both platforms:toJSONis honored (aDatebecomes its ISO string),undefinedproperties are dropped, and non-finite numbers becomenull. - The internal wire form
{"__protect_ffi_bigint__": "<digits>"}is reserved: acast_as: 'json'plaintext consisting of exactly that single-key shape (with an in-range i64 decimal string) is read as a bigint at the boundary and rejected for json columns. Nested occurrences of the key inside a larger document are unaffected.
const ciphertext = await addon.encrypt(client, {
plaintext: 9007199254740993n, // beyond Number.MAX_SAFE_INTEGER — stays exact
column: 'score',
table: 'users',
})
const decrypted = await addon.decrypt(client, { ciphertext })
// decrypted === 9007199254740993n (a JS bigint)[!WARNING] Breaking change:
decrypt/decryptBulk/decryptBulkFalliblenow ALWAYS return a JSbigintforcast_as: 'bigint'columns — even for values that fit in a JS number. Previous releases returned anumber, silently losing precision beyondNumber.MAX_SAFE_INTEGER(2^53 - 1). Code comparing or doing arithmetic on decrypted bigint columns must be updated (e.g.decrypted === 123ninstead ofdecrypted === 123, orNumber(decrypted)when the value is known to be small).
Errors
Both entries throw an ordinary JS Error with a stable code, set in Rust
from the error variant. There is no wrapper class and nothing to unwrap.
Not every failure has a code — the ones wrapping a cipherstash-client error
arrive without the field. 'UNKNOWN' is the name for that case, not a value
Rust emits.
TypeScript types a catch variable as unknown, so you narrow once. That
needs nothing from this package:
try {
await addon.encryptQuery(client, opts)
} catch (err) {
if (err instanceof Error && 'code' in err && err.code === 'INVALID_JSON_PATH') {
// handle JSON path mistakes
}
throw err
}To carry the code around as a typed value, use isProtectErrorCode. Checking
the value rather than the field's presence matters: Node sets code on its own
errors, so a bare err?.code read would let an ECONNRESET pass for one of
these.
import {
isProtectErrorCode,
type ProtectErrorCode,
} from '@cipherstash/protect-ffi'
function errorCode(err: unknown): ProtectErrorCode | undefined {
const { code } = err as { code?: unknown }
return isProtectErrorCode(code) ? code : undefined
}The codes themselves are PROTECT_ERROR_CODES in src/errors.ts, which
src/errorCodes.test.ts checks against the #[diagnostic(code(..))]
attributes on Error in crates/protect-ffi/src/lib.rs — the one place a code
is decided.
Available Scripts
In the project directory, you can run:
Since this package was absorbed into the cipherstash/stack monorepo, cargo is deliberately kept off the default build and test paths: the root pnpm test reaches every package, and a cargo process on that path would put a Rust toolchain on every contributor's machine. The Rust work lives behind its own scripts, listed below. packages/protect-ffi/src/lintWiring.test.ts enforces the split.
pnpm run build
Compiles the TypeScript to lib/ with tsc. This is the package main, and it is generated — a workspace consumer resolves an empty package until this has run. It does not build the native addon; use build:native for that.
pnpm run build:native
Builds the Node addon (index.node) from source, generating a release build with cargo --release.
Additional cargo build arguments may be appended. Note the absence of a -- separator — pnpm forwards -- verbatim (npm strips it), and these scripts end in > cargo.log, so a separated flag lands after the redirect and cargo rejects it as a positional argument. For example, to enable a cargo feature:
pnpm run build:native --feature=beetlepnpm run debug
Similar to build:native but generates a debug build with cargo.
pnpm run cross
Similar to build:native but uses cross-rs to cross-compile for another platform. Use the CARGO_BUILD_TARGET environment variable to select the build target. pnpm run zigbuild does the same via cargo-zigbuild, which is what the release matrix uses to pin glibc.
pnpm run build:wasm
Builds the wasm32-unknown-unknown output into dist/wasm/ with wasm-pack, then inlines the .wasm as base64 for the ./wasm-inline entry. Needs the wasm32 target and wasm-pack (both supplied by mise install in this directory).
pnpm test
Typechecks the TypeScript, runs the unit tests (tsc + vitest), and lints and format-checks the TypeScript. No cargo, by design — see above.
Note: this does not run the integration tests either. For those, see below.
pnpm run test:cargo
The Rust half of the test suite: cargo test plus cargo fmt --check. Run by a path-filtered root workflow, not by pnpm test.
pnpm run test:typecheck:wasm
Typechecks type-tests/ against the wasm-bindgen-generated declarations in dist/wasm/. Needs build:wasm and build to have run first, which is why it is not part of pnpm test — that has to pass in a fresh clone with no dist/.
mise run lint:rust
Every Rust check CI gates on: clippy against the host target, clippy against wasm32-unknown-unknown, and cargo fmt --check. pnpm test leaves all of these out, so run this before pushing a Rust change.
The wasm arm needs the target installed once:
rustup target add wasm32-unknown-unknownmise run fmt fixes what the formatting check reports.
Project Layout
The directory structure of this project is:
protect-ffi/
├── Cargo.toml
├── README.md
├── integration-tests/
├── lib/
├── src/
| ├── index.mts
| └── index.cts
├── crates/
| └── protect-ffi/
| └── src/
| └── lib.rs
├── platforms/
├── docs/
├── scripts/
├── package.json
└── target/| Entry | Purpose |
| -------------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
| Cargo.toml | The Cargo manifest file, which informs the cargo command. |
| README.md | This file. |
| integration-tests/ | The directory containing integration tests. |
| lib/ | The directory containing the generated output from tsc. |
| src/ | The directory containing the TypeScript source files. |
| index.mts | Entry point for when this library is loaded via ESM import syntax. |
| index.cts | Entry point for when this library is loaded via CJS require. |
| crates/ | The directory tree containing the Rust source code for the project. |
| lib.rs | Entry point for the Rust source code. |
| platforms/ | The directory containing distributions of the binary addon backend for each platform supported by this library. |
| docs/ | JSONB integration docs and migration notes. |
| scripts/ | Build helper scripts (e.g. WASM inlining). |
| package.json | The npm manifest file, which informs the npm command. |
| target/ | Binary artifacts generated by the Rust build. |
Integration tests
Integration tests live in the ./integration-tests directory.
These tests use the local build of Rust and JavaScript artifacts to test @cipherstash/protect-ffi as API consumers would.
These tests rely on:
- CipherStash to be configured (via
.tomlconfig or environment variables), and - Environment variables for Postgres to be set
Example environment variables:
CS_CLIENT_ID=
CS_CLIENT_KEY=
# The integration tests read CS_CLIENT_ACCESS_KEY; the library itself reads CS_ACCESS_KEY
CS_CLIENT_ACCESS_KEY=
CS_WORKSPACE_CRN=
# Must match the port used by the integration-test setup (mise.toml)
PGPORT=5436
PGDATABASE=cipherstash
PGUSER=cipherstash
PGPASSWORD=password
PGHOST=localhostTo run integration tests:
mise setup
mise test:integrationYou can also run the integration tests in "watch" mode:
mise test:integration --watchBy default lock context tests are not included because invalid lock contexts fire security warnings in ZeroKMS. To include these, run:
mise test:integration:allReleasing
This package was absorbed into the cipherstash/stack
monorepo, and its releases now run on that repository's
Changesets flow rather than a
workflow_dispatch trigger. There is no longer a version to pick by hand: the
wrapper and its six platform packages share a Changesets fixed group, so a
release bumps all seven in lockstep.
Write a changeset (pnpm changeset at the repo root) naming
@cipherstash/protect-ffi; the fixed group propagates the bump to the platform
packages. The changeset body becomes the CHANGELOG.md entry — Changesets owns
that file now, so the [Unreleased] heading and the version npm lifecycle
hook that used to promote it are gone.
npm trusted publishing for all seven packages names cipherstash/stack and the
repository-root release.yml, so a changeset here releases like any other
package. The temporary guard that blocked one during the cutover window, and the
.md.deferred convention for parking one behind it, are both gone.
No FFI release has run from this repository yet — 0.31.0 was published from the old one. Until the first one lands, the path is configured rather than proven.
The previous repository's GitHub Actions workflows were deposited under
.github/ in this directory by the subtree import and kept as the reference for
that port. They are gone: the six-platform build matrix now lives in the
repository-root .github/workflows/_build-ffi-artifacts.yml, with
ffi-preflight.yml as its manually-dispatched dry run. Nothing under a package
directory is ever executed by GitHub, which reads workflows from the repository
root alone — src/lintWiring.test.ts fails if one comes back.
Learn More
Learn more about:
