Backup Data SDK for JavaScript / TypeScript

JavaScript/TypeScript SDK for Backup Data: workspace-scoped incremental, content-addressed backup and restore with SIWE / email / API-key authentication and optional client-side encryption.

This is a 1:1 port of the Go SDK. The encryption, chunking, and canonical tree pipelines are byte-compatible with the Go SDK and the existing backend: both SDKs produce identical chunk hashes, tree hashes, and key material for the same input, and each can restore (and incrementally build on) snapshots created by the other.

Overview

• Backup pipeline: scan → chunk → deduplicate → (optional) compress → (optional) encrypt → upload packs → create snapshot.
• Restore pipeline: download → (optional) decrypt → (optional) decompress → verify integrity → reassemble files.

Features

• Auth: SIWE (EIP-4361), email/password login + verification, and lh_-prefixed API key bearer tokens.
• Workspace-scoped backup/restore — set a default workspace on the client or override per call.
• FastCDC content-defined chunking for stable deduplication (identical gear table and cut points as the Go SDK).
• Pack uploads to aggregate chunks for efficient object storage.
• Bloom-assisted dedup using server-provided filters.
• Incremental backup using file metadata to reuse unchanged chunk lists.
• Canonical directory trees for cross-language compatibility.
• Optional client-side encryption: passphrase-protected tenant key (TMK, Argon2id + AES-256-GCM keyfile), per-snapshot data encryption key (DEK) wrapped client-side, per-object keys from HKDF-SHA256 over the DEK and content hash.
• Snapshot lifecycle: list, inspect, create, delete, prune.
• TMK rotation: re-wrap every snapshot's DEK under a new TMK without re-encrypting any data blobs.
• Workspace & member management + user profile / identity / API-key APIs.
• Restore verification against expected chunk hashes.

Requirements

• Node.js 18 or newer.

Installation

npm install @backupdata/js-sdk

Create a local npm package

npm run pack

This builds the SDK and writes an installable tarball such as backupdata-js-sdk-0.1.0.tgz in the repository root. Install it in another project with:

npm install /path/to/backupdata-js-sdk-0.1.0.tgz

Quick start

import { BackupClient } from "@backupdata/js-sdk";

const client = new BackupClient({
  apiKey: "lh_...",
  workspaceId: "550e8400-e29b-41d4-a716-446655440000",
});

const snapshot = await client.backup(["/path/to/data"], {
  description: "nightly",
});

await client.restore(snapshot.snapshotId, "/path/to/restore-target");

The Backup Data API endpoint is built into the SDK — you do not configure it. BackupClient always targets https://api.backupdata.io.

Encrypted backup

import { BackupClient, generateKeyfile } from "@backupdata/js-sdk";

// One-time: create a passphrase-protected keyfile holding the tenant key.
generateKeyfile("/secure/keyfile.json", "my passphrase");

const client = new BackupClient({
  apiKey: "lh_...",
  workspaceId: "...",
});

const encryption = {
  keyfilePath: "/secure/keyfile.json",
  passphrase: "my passphrase",
};

const snap = await client.backup(["/path/to/data"], { encryption });
await client.restore(snap.snapshotId, "/restore/here", { encryption });

Keyfiles are interchangeable between the Go and JS SDKs — a keyfile created by either SDK opens in the other, and either SDK can restore the other's encrypted snapshots.

DEK reuse across versions. Because deduplication is workspace-scoped and content-addressed, every encrypted snapshot in a workspace must share one data encryption key (DEK) — otherwise a later snapshot that reuses deduplicated objects encrypted under an earlier DEK cannot decrypt them. So the JS SDK inherits the DEK from the parent snapshot and only mints a fresh one for the first encrypted snapshot in a workspace. This makes repeated and incremental encrypted backups restorable. Two consequences:

• Encrypted backups into a workspace must use the keyfile that created it; a mismatched keyfile is rejected with a clear error instead of silently producing an unrestorable snapshot.
• Dedicate a workspace to a single encryption mode. Don't mix encrypted and unencrypted backups in the same workspace, and start encrypted backups in a fresh workspace.

Note: DEK inheritance is currently a JS-SDK behavior. The upstream Go SDK still mints a fresh DEK per snapshot, so a second Go-driven encrypted backup into the same workspace remains unrestorable until the equivalent change lands there. The fix changes no chunk/tree hashes, so cross-SDK content compatibility is unaffected.

SIWE authentication

const client = new BackupClient({
  privateKey: "0x...", // or address + signMessage callback
  workspaceId: "...",
});
await client.authenticate();

Module layout

Mirrors the Go SDK package layout under src/:

• client: high-level SDK client (BackupClient) and source-ID persistence.
• api: typed HTTP transport + auth flows + S3 upload/download helpers.
• pipeline: backup/restore/rotation workflows.
• types: API payload and SDK option types.
• errors: structured ApiError helpers.
• encrypt: keyfile/TMK/DEK/HKDF/AES-GCM encryption helpers.
• chunk: FastCDC chunking + SHA-256 hashing.
• dedup: bloom filter dedup support.
• tree: canonical tree serialization and hashing.
• codec: compression/decompression helpers (zstd).
• pool: batching and bounded-concurrency helpers.

Compatibility testing

npm test builds the SDK and runs two suites:

1. Cross-SDK vectors (test/compat.test.js + test/vectors.json): test vectors generated by the Go SDK — FastCDC chunk boundaries and hashes (including 24 MiB at default 1/4/16 MiB options), canonical tree JSON and hashes (unicode/emoji names, zero values, empty trees), HKDF-derived keys, Go-wrapped DEKs, Go-encrypted objects, Argon2id KEKs, a Go-generated keyfile, bloom filter membership behavior, deterministic Ethereum signatures, and Go-compressed zstd blobs — all replayed through the JS SDK and asserted byte-identical.
2. Round-trip (test/roundtrip.test.js): encrypted backup → restore → directory diff against an in-memory mock backend, plus an incremental backup asserting an unchanged root tree hash.
3. Encryption + dedup (test/encryption.test.js): runs against a content-addressed mock (first-write-wins + bloom filter) so deduplication actually happens. Covers repeated and incremental encrypted backups with DEK inheritance (each restored and diffed), and asserts a mismatched keyfile is rejected with a clear error.

Compatibility notes

Behavioral quirks intentionally preserved from the Go SDK so that both implementations always produce identical trees:

• Tree node mtimes are UTC with millisecond precision, truncated (not rounded) from the filesystem timestamp.
• File/dir modes store only the permission bits (mode & 0o777).
• A file's content field is omitted when the file has no chunks (empty files), and empty directories are omitted from their parent tree.
• Tree nodes sort by UTF-8 byte order of their names (not UTF-16), and an empty node list serializes as {"nodes":null}.