Loro Roost Node native binding

Experimental Node package backed by the public [email protected] Rust crate. napi-rs builds a .node library; the JavaScript client loads it in one Worker and uses the existing SQLite, signatures, ID ordering and transaction implementation. A separately supplied roost-node-owner executable is not part of this entry point.

This is an experimental package. Its local history API has been tested with NodeLodyHistory from @loro-dev/[email protected]. The browser package and subprocess client remain separate entry points. Remote Streams operations use the Rust registry, Link, Executor and HistoryReader. Authentication and payload crypto are asynchronous host callbacks; production key storage remains host owned.

Build and verify

Use a stable Rust toolchain and Node 22.14+ with the pinned pnpm version:

cd node/native
pnpm install --frozen-lockfile
RUSTUP_TOOLCHAIN=stable pnpm build
pnpm test
pnpm check
pnpm format:check
cargo +stable fmt --check
cargo +stable test --locked
cargo +stable clippy --locked --all-targets -- -D warnings

binding.cjs and binding.d.ts are generated by napi-rs. Local builds load an adjacent .node; published installs load the matching optional platform package. The native module exposes a blocking local DTO entry and an asynchronous sync entry; use the asynchronous root entry unless your embedding already owns a dedicated Worker and controls its request/callback lifetime.

import { RoostNativeClient } from "@loro-dev/roost-node";

const client = new RoostNativeClient({
  dbPath: "/absolute/path/to/history.db",
  seed: securelyLoadedSeed,
  allowedOwners: admittedAuthorPublicKeys,
  maxQueuedRequests: 32,
  maxQueuedBytes: 4 * 1024 * 1024,
});
await client.ready;
try {
  const stream = client.stream("conversation-1");
  const { updates } = await stream.create([], '{"text":"hello"}');
  const result = await stream.readTurn(updates[0].turnId);
} finally {
  await client.close();
}

The seed and admission list above are host-provided values, not keys generated or stored by this package. ready.owner is the signing author's public key; it does not identify a separate engine process. Tests only use synthetic seeds and temporary files.

Streams synchronization

Supply auth and crypto when constructing a client if those modes are selected. auth({remoteId, reason, signal}) returns a nonempty bearer token for the current cycle. crypto.encrypt/decrypt({remoteId, bytes, signal}) return transformed bytes; the Rust transport adds/removes the Roost encrypted envelope. The host implements the actual cryptography and owns keys. No plaintext fallback is attempted.

With an open client, register a credential-free endpoint and an explicit remote generation supplied by your application's stream lifecycle:

const { id } = await client.registerRemote({
  name: "cloud",
  url: streamsEndpointUrl,
  physicalStream: physicalStreamName,
  generation: hostConfirmedGeneration,
});
const sync = client.stream("conversation-1").sync(id, {
  partial: false,
  encryption: "host",
  authentication: "host",
});
await sync.ensureRemote();
const upload = await sync.upload({ maxSend: 100n });
const download = await sync.download({ pageLimit: 10n });

listRemotes() and setRemoteEnabled(id, enabled) use the same durable SQLite store as history. Full uploads discover committed local writes and imported updates through Rust's persisted event checkpoint. maxSend bounds sends, not that discovery catch-up. Upload reports distinguish sent, confirmed and transportError; an empty send list is not proof of remote completeness.

For partial replicas, select partial: true. fetchWindow({budgetBytes, pageLimitBytes}) retrieves a bounded reverse-history window, and completePrefixes(limits) recovers missing prefixes. Pass the original returned fetchedRange[1] to catchUpAt(tail) before live downloads. Partial uploads must supply 1..1000 exact turnIds; absent ancestors remain reported in waitingParents. Seals verify their own content without claiming that an absent ancestor has been retrieved or verified.

Callbacks execute outside SQLite transactions. Sync runs on Rust blocking threads, leaving the Worker event loop available to receive callback replies. Replies bypass the ordinary request queue. Reentering this client from its auth or crypto callback rejects callbackReentry; calls from unrelated producers may queue. A 401/403 requests auth reason unauthorized on the next cycle. Credentials are transient, excluded from registry rows and diagnostics.

Host callbacks have a 30s default deadline, configurable via callbackTimeoutMs to a positive value <=30000. Expiry aborts the callback's signal and rejects auth or fails crypto; late results are ignored by callback identity. Rust has a separate 35s backup wait. A persisted upload package retries its exact saved ciphertext after an unconfirmed append, even if host keys change.

Lifetime and values

One client owns one Worker and one Arc<SqliteStorage>; all its stream handles share that connection. The host must share a client for each database and stop message producers before shutdown. There is no cross-process singleton or automatic database ownership arbitration.

Calls are FIFO with one active native call and bounded waiting request count and bytes. Inputs are copied before enqueueing. close() refuses new requests, drains accepted calls, closes SQLite and waits for Worker exit. It is idempotent. AbortSignal can remove a queued request; after dispatch the operation continues to its result. Worker loss gives the dispatched call unknownOutcome and queued calls notDispatched. Local writes are never automatically retried; explicit sync retries reuse Rust's durable upload work.

Worker failure errors retain local diagnostic evidence in a non-enumerable cause. Dispatched unknownOutcome and queued notDispatched errors reference the same terminal failure. client.failure exposes that failure after an initialization exception, Worker fault or unexpected exit, even when no request was active. An unexpected exit includes an Error cause with exitCode; later closed requests retain the terminal cause. Error codes remain the authority for request outcomes; inspecting a cause does not authorize replay or rollback.

Binding faults copy only Error name, message, stack and a bounded cause chain across the local Worker boundary. Embeddings decide how to record those fields; they may include local paths, so do not forward raw causes to remote peers or include them in conversation history. The SDK does not add logging or telemetry.

Close has a 30s default drain deadline (closeTimeoutMs, positive <=30000). On expiry the dispatched request becomes unknownOutcome, queued work becomes notDispatched, and close fails closeTimeout after actual Worker exit. Native work stops cooperatively between operations, pending crypto is released, and SQLite closes after the active native task finishes. An in-flight HTTP request can finish later than the drain deadline: stopping a Worker does not roll back or forcibly interrupt Rust. The public crate's HTTP timeouts still govern that RPC.

Turn IDs and hashes are Uint8Array; counters and cursors are u64 bigint. Content and replacement values are canonical Rust JSON strings: do not parse them through JavaScript number if their exact u64/i64/f64 representation matters. The private native DTO conversion is separate from Roost's import/export wire format. Local success means a SQLite commit, not delivery to a remote service.

Unlike an independent process, a Node native module shares its process's failure boundary. The Worker protects the JS event loop from blocking database calls, not the containing process from a native crash.

Local tests use real SQLite, Ed25519 seals, AES-GCM host callbacks and a scripted loopback Streams endpoint. They are binding verification, not acceptance against a deployed Streams service or a packaged Electron GUI.

Prepare and publish

Starting with 0.1.1, the main package contains the JavaScript client, Worker, loader and types. Six optional platform packages each contain one native library. Exact versions and os/cpu/Linux libc fields let npm download only the matching binary. Consumers install @loro-dev/roost-node as before and need neither Rust nor a separate executable. Version 0.1.0 retains its original all-in-one layout.

| Platform | CPU | | ------------------------ | ---------- | | macOS | arm64, x64 | | Windows (MSVC) | arm64, x64 | | Linux (GNU, glibc 2.35+) | arm64, x64 |

Alpine/musl is not included. GNU/Linux builds vendor OpenSSL, avoiding a runtime dependency on the host's libssl/libcrypto.

The Node native release preparation workflow still builds on the same six matching runners. It tests real SQLite/signatures/sync on Node 24.14.0 and 22.14.0 and uses a loopback registry to prove a clean npm consumer downloads only its platform tarball, with install scripts disabled. The workflow collects only matching-source, SHA-256-checked artifacts and produces roost-node-<version>-ready.tgz. CI builds, checks and previews; publication remains an explicit user action. Package splitting adds no build targets.

Extract the ready artifact into an empty directory. It contains main/, npm/<platform>/, a complete hash manifest and a private orchestration package. Only Node 22.14+ and npm are needed:

npm run release:check
npm run release:preview
# Explicitly publish six platform packages, then the main package:
npm run release:publish

The check and preview commands never upload anything. prepublishOnly only validates the main package; it never publishes child packages. The explicit publish command verifies the whole bundle, packs all seven packages, checks registry integrity before uploading, publishes missing platforms first and publishes the main package only when every platform is available. A retry skips only already-published versions with identical tarball integrity. If npm is still processing a platform, rerun the same command using the unchanged bundle. The npm publisher needs access to the main name and all six suffixed names. Seven npm publications are not atomic; do not publish main/ separately before the platform packages exist.

To assemble downloaded native-* artifacts manually from the corresponding source revision:

npm run release:prepare -- /path/to/merged-artifacts /path/to/ready-directory EXACT_GIT_SHA
cd /path/to/ready-directory
npm run release:preview

Package names are @loro-dev/roost-node-<platform>, for example @loro-dev/roost-node-darwin-arm64 and @loro-dev/roost-node-linux-x64-gnu. Preparation generates them from one target table; the source checkout deliberately does not depend on unpublished platform versions. Its local build still works without a registry publication. The platform-packages command is a development utility; release preparation is the authoritative complete bundle.

For a local package smoke check, build first and run npm run packed:smoke. Keep the client, Worker, native loader and matching platform package in an Electron package, or stage its binary adjacent to the generated loader. Test the packed runtime. Electron ASAR packaging must unpack the native module and preserve the Worker entry's relative paths.

In the source checkout, scope, execution records and verification limits: binding plan.