@loro-dev/flock-wasm

WASM bindings for flock-rs — a distributed, conflict-free key-value store built on CRDT (Conflict-free Replicated Data Type) principles.

Flock enables multiple replicas to independently read and write data, merge changes without coordination, and converge deterministically to the same state. It is designed for offline-first, peer-to-peer, and decentralized applications where network partitions are expected and no central authority is required.

Table of Contents

• Install
• Core Concepts
• Quick Start
• Runtime Modes and Buffering
• API Reference
  • Top-Level Exports
  • Flock Constructors
  • Identity and Diagnostics
  • Data Operations
  • Multi-Value Register (MVR)
  • Scan Operations
  • Replication and Import/Export
  • Hooks API
  • Events and Subscriptions
  • Debounced Event Batching
  • Transactions
  • Binary File Export/Import
• Type Reference
• Conflict Resolution Deep Dive
• Version Vectors Explained
• Best Practices
• Common Pitfalls
• Build
• Test
• License

Install

npm add @loro-dev/flock-wasm
# or
pnpm add @loro-dev/flock-wasm
# or
yarn add @loro-dev/flock-wasm

Core Concepts

CRDT with Last-Writer-Wins (LWW) Semantics

Every entry in Flock carries a clock consisting of three components:

1. Physical time — wall-clock timestamp in milliseconds.
2. Logical counter — monotonic counter to break ties at the same physical time.
3. Peer ID — unique identifier of the writer.

When two replicas have conflicting values for the same key, Flock resolves deterministically:

1. Higher physicalTime wins.
2. On tie, higher logicalCounter wins.
3. On tie, lexicographically greater peerId wins.

All replicas applying the same rule to the same set of entries converge to the identical value — no coordination needed.

Keys and Values

• Keys are arrays of JSON-serializable parts: ["users", 42, "name"]. Keys are encoded into a memcomparable binary format, enabling efficient sorted storage and prefix-based range scans.
• Values are any JSON-serializable type: strings, numbers, booleans, null, arrays, or nested objects.
• Metadata is an optional Record<string, unknown> attached to each entry, useful for signatures, policy hints, or audit information.

Tombstones

Deletions create tombstone entries rather than removing data. A tombstone carries a clock but no data. Tombstones:

• Are visible in scan() results (with value: undefined).
• Participate in conflict resolution — a delete with a higher clock wins over an older write.
• Can be pruned during export via pruneTombstonesBefore.
• Ensure that "deleted" and "never existed" are distinguishable via getEntry().

Version Vectors

A version vector is a compact summary of the visible state: { [peerId]: { physicalTime, logicalCounter } }. Two types exist:

• Exclusive version (version()): Only includes peers that currently own at least one visible entry. Use this as the baseline for incremental sync.
• Inclusive version (inclusiveVersion()): Tracks max seen clocks across all peers ever encountered. Use this for completeness checks.

Quick Start

import { Flock } from "@loro-dev/flock-wasm";

// Create two independent replicas
const alice = new Flock("alice");
const bob = new Flock("bob");

// Alice writes some data
alice.put(["todos", 1], { title: "Write docs", done: false });
alice.put(["todos", 2], { title: "Ship feature", done: false });

// Export Alice's state and import into Bob
const snapshot = alice.exportJson();
bob.importJson(snapshot);

// Bob sees Alice's data
console.log(bob.get(["todos", 1])); // { title: "Write docs", done: false }

// Both edit concurrently
alice.put(["todos", 1], { title: "Write docs", done: true });
bob.put(["todos", 1], { title: "Write better docs", done: false });

// Merge resolves conflict via LWW (higher timestamp wins)
alice.merge(bob);
bob.merge(alice);

// Both converge to the same value
console.log(alice.get(["todos", 1])); // same result
console.log(bob.get(["todos", 1]));   // same result

Runtime Modes and Buffering

All Flock constructors (new, fromJson, fromFile) use buffered mode by default:

• Memtable enabled: Writes are accumulated in an in-memory buffer.
• Flushed on commit: Buffered writes are flushed when txnCommit() is called, when the storage transaction commits, or when exporting file bytes.
• Implication: For strongest durability semantics, explicitly commit transactions or call exportFile() more frequently.

API Reference

Top-Level Exports

import {
  Flock,
  encodeVersionVector,
  decodeVersionVector,
  type VersionVector,
  type VersionVectorEntry,
  type ExportBundle,
  type ScanOptions,
  type ScanRow,
  type ScanBound,
  type EventBatch,
  type Event,
  type EventPayload,
  type EntryInfo,
  type EntryClock,
  type ExportRecord,
  type MetadataMap,
  type Value,
  type KeyPart,
  type ExportPayload,
  type ExportHookContext,
  type ExportHooks,
  type ImportPayload,
  type ImportHookContext,
  type ImportAccept,
  type ImportSkip,
  type ImportDecision,
  type ImportHooks,
  type PutPayload,
  type PutHookContext,
  type PutHooks,
  type PutWithMetaOptions,
} from "@loro-dev/flock-wasm";

encodeVersionVector(vector: VersionVector): Uint8Array

Encodes a version vector into a compact binary representation.

• Input: A VersionVector object mapping peer IDs to { physicalTime, logicalCounter }.
• Output: Uint8Array with VEVE magic prefix, ULEB128-encoded timestamps, and delta-compressed entries.
• Use case: Persist or transmit version vectors efficiently. The binary format is deterministic — same logical content always produces identical bytes.
• Skips: Entries with invalid peer IDs (non-string or >= 128 UTF-8 bytes) or non-finite timestamps are silently ignored.

decodeVersionVector(bytes: Uint8Array): VersionVector

Decodes a binary version vector back into a VersionVector object.

• Input: Uint8Array produced by encodeVersionVector.
• Output: VersionVector object.
• Compatibility: Handles both the current VEVE-prefixed format and the legacy format (no magic prefix) transparently.
• Throws: TypeError if peer IDs are invalid or timestamps are non-monotonic.

Flock Constructors

new Flock(peerId?: string)

Creates a new empty Flock instance.

• peerId (optional): A UTF-8 string under 128 bytes identifying this peer. If omitted, a cryptographically random 64-character hex string is generated.
• Returns: A new Flock instance in buffered mode.
• Throws: TypeError if peerId is not a valid string or exceeds 127 UTF-8 bytes.
• Side effects: None. The instance starts with empty state and no version history.

const named = new Flock("peer-alice");
const anonymous = new Flock(); // random 64-char hex ID

Flock.fromJson(bundle: ExportBundle, peerId: string): Flock

Creates a new Flock instance pre-populated with state from a JSON export bundle.

• bundle: An ExportBundle obtained from exportJson() on another instance.
• peerId: The peer ID for the new instance (required).
• Returns: A new Flock with merged state from the bundle.
• Throws: If the bundle format is invalid or import fails.
• Use case: Bootstrap a new replica from a snapshot received over the network.

const snapshot = source.exportJson();
const replica = Flock.fromJson(snapshot, "new-peer");

Flock.fromFile(bytes: Uint8Array, peerId?: string): Flock

Creates a new Flock instance from a binary file export. This is the fastest way to load state — the pre-built B+Tree pages are used directly without re-parsing or re-indexing (O(1) open time vs. O(n log n) for fromJson).

• bytes: Uint8Array produced by exportFile().
• peerId (optional): Peer ID for the new instance. Random if omitted.
• Returns: A new Flock with full state restored from the binary format.
• Throws: If the binary format is invalid (wrong magic bytes, corrupted pages, CRC mismatch).
• Use case: Fast cold-start, restore from disk, or load a persisted checkpoint. Prefer this over fromJson when loading full state.

const bytes = fs.readFileSync("state.db");
const restored = Flock.fromFile(new Uint8Array(bytes), "restored-peer");

Identity and Diagnostics

peerId(): string

Returns the current peer ID of this Flock instance.

• Returns: The peer ID string (always valid UTF-8, under 128 bytes).
• Throws: TypeError if the internal FFI returns an unexpected value (should not happen in normal operation).
• Note: For anonymous instances, returns the auto-generated 64-character hex string.

setPeerId(peerId: string): void

Changes the peer ID for subsequent operations.

• peerId: New peer ID (UTF-8, under 128 bytes).
• Effect: All future put, delete, and putWithMeta calls will be attributed to the new peer ID.
• Does NOT retroactively change previous entries — those remain attributed to the original peer.
• Throws: TypeError if the peer ID is invalid.

flock.setPeerId("peer-a");
flock.put(["k"], "by-a");   // attributed to "peer-a"
flock.setPeerId("peer-b");
flock.put(["k"], "by-b");   // attributed to "peer-b", overwrites "by-a" if clock is higher

checkInvariants(): void

Validates the internal CRDT state for consistency.

• Effect: Exercises version(), inclusiveVersion(), and scan() to check structural integrity.
• Throws: If any internal invariant is violated.
• Use case: Debugging and testing. Not needed in production.

getMaxPhysicalTime(): number

Returns the maximum physical timestamp across all entries.

• Returns: A number (milliseconds). Returns 0 for an empty Flock.
• Use case: Determine a safe threshold for tombstone pruning. For example, pruneTombstonesBefore: flock.getMaxPhysicalTime() - 86400000 prunes tombstones older than 24 hours.
• Note: This value only increases monotonically during the lifetime of the instance.

Data Operations

put(key: KeyPart[], value: Value, now?: number): void

Stores a value at the specified key using LWW semantics.

• key: Array of JSON-serializable parts (e.g., ["users", 42]).
• value: Any JSON-serializable value (string, number, boolean, null, array, object).
• now (optional): Explicit physical timestamp in milliseconds. If omitted, Date.now() is used.
• Effect: Creates or overwrites the entry at key. If the key already exists, the new entry wins only if its clock is higher than the existing entry's clock.
• Event: Emits a change event to all subscribers (subject to debounce/transaction batching).
• Throws: If the key or value is not JSON-serializable.

Important: put always succeeds locally — the LWW comparison happens during merge with remote state. Locally, put always overwrites because the local clock is guaranteed to advance.

flock.put(["config", "theme"], "dark");
flock.put(["scores", 1], 100, Date.now());
flock.put(["nested"], { a: { b: [1, 2, 3] } });

set(key: KeyPart[], value: Value, now?: number): void

Alias for put(). Identical behavior.

putWithMeta(key: KeyPart[], value: Value, options?: PutWithMetaOptions): void | Promise<void>

Stores a value with optional metadata and/or transform hooks.

• key: Key array.
• value: Value to store.
• options.metadata (optional): A MetadataMap (Record<string, unknown>) to associate with this entry. Useful for signatures, audit trails, or policy tags.
• options.now (optional): Explicit timestamp.
• options.hooks.transform (optional): An async function (context, payload) => payload | void that can modify the data and metadata before storage. If provided, the method returns a Promise<void>.
• Returns: void if no hooks; Promise<void> if hooks are present.
• Throws: TypeError if the transform hook returns a payload without a data field.
• Event: Emits a change event after the (possibly async) operation completes.

// Simple metadata
flock.putWithMeta(["doc", 1], "content", {
  metadata: { author: "alice", signature: "abc123" },
});

// With transform hook (async)
await flock.putWithMeta(["doc", 1], "content", {
  metadata: { sig: "pending" },
  hooks: {
    transform: async (ctx, payload) => {
      payload.metadata = { ...payload.metadata, sig: await sign(payload.data) };
      return payload;
    },
  },
});

delete(key: KeyPart[], now?: number): void

Deletes the value at the specified key by creating a tombstone.

• key: Key array.
• now (optional): Explicit timestamp.
• Effect: Creates a tombstone entry. get(key) will return undefined after deletion.
• Important: The tombstone participates in conflict resolution. A delete with a lower clock than an existing write will be ignored during merge. A delete with a higher clock will win.
• No-op on missing keys: Deleting a key that has never been written is a no-op — no tombstone is created and no event is emitted. Deleting an already-deleted key is also a no-op (the existing tombstone is kept).
• Event: Emits a change event with value: undefined when an existing entry is deleted.

flock.put(["temp"], "data");
flock.delete(["temp"]);
flock.get(["temp"]); // undefined

get(key: KeyPart[]): Value | undefined

Retrieves the current value at the specified key.

• key: Key array.
• Returns: The stored value, or undefined if the key does not exist or has been deleted.
• Note: Does not distinguish between "never set" and "deleted". Use getEntry() for that distinction.
• Graceful: If the key array cannot be decoded (e.g., contains non-serializable types), returns undefined instead of throwing.

flock.put(["k"], 42);
flock.get(["k"]);         // 42
flock.get(["missing"]);   // undefined

getEntry(key: KeyPart[]): EntryInfo | undefined

Retrieves the full entry including data, metadata, and clock information.

• key: Key array.
• Returns: An EntryInfo object { data?, metadata, clock }, or undefined if the key has never been written.
• Tombstone distinction: For deleted keys, returns an object with data omitted but clock and metadata present. For never-set keys, returns undefined.
• Metadata: Always present as an object (defaults to {} when no metadata was stored).
• Use case: Auditing (who wrote when), tombstone detection, signature verification.

flock.putWithMeta(["k"], "value", { metadata: { author: "alice" } });
const entry = flock.getEntry(["k"]);
// {
//   data: "value",
//   metadata: { author: "alice" },
//   clock: { physicalTime: 1700000000000, logicalCounter: 0, peerId: "alice" }
// }

flock.delete(["k"]);
const tombstone = flock.getEntry(["k"]);
// {
//   metadata: {},
//   clock: { physicalTime: 1700000000001, logicalCounter: 0, peerId: "alice" }
// }
// Note: `data` field is absent (not `undefined` — it is omitted)

flock.getEntry(["never-set"]); // undefined

Multi-Value Register (MVR)

MVR is a pattern built on top of Flock's LWW key-value store that allows concurrent writes from different peers to coexist instead of one winning. It works by appending the value to the key and storing true as a marker.

putMvr(key: KeyPart[], value: Value, now?: number): void

Stores a scalar value using the MVR pattern.

• key: Base key array (the MVR "slot").
• value: Must be a scalar JSON value: string, number, or boolean. Objects, arrays, and null are rejected.
• now (optional): Explicit timestamp.
• Effect: Deletes all existing MVR entries under the base key (where value === true), regardless of which peer wrote them, then writes true at [...key, value].
• Throws: TypeError if value is null, an array, or an object.
• Event: Emits delete events for removed entries and a put event for the new entry.

How it works internally:

putMvr(["mv"], "one") →
  scan prefix ["mv"] and delete all entries where value === true
  put(["mv", "one"], true)

getMvr(key: KeyPart[]): Value[]

Retrieves all concurrent MVR values for a base key.

• key: Base key array.
• Returns: Array of scalar values. Empty [] if no values exist.
• Behavior: Scans all entries with prefix key, filters for entries at depth key.length + 1 with value === true, and extracts the appended value part.

// Single writer
flock.putMvr(["color"], "red");
flock.getMvr(["color"]); // ["red"]

// Concurrent writers merge
const a = new Flock("a");
const b = new Flock("b");
a.putMvr(["color"], "red");
b.putMvr(["color"], "blue");
a.merge(b);
a.getMvr(["color"]); // ["red", "blue"] (order may vary)

// Calling putMvr replaces ALL values (not just the local peer's)
a.putMvr(["color"], "green");
a.getMvr(["color"]); // ["green"] — both "red" and "blue" were deleted

Scan Operations

scan(options?: ScanOptions): ScanRow[]

Scans entries within the specified key range and/or prefix.

• options.start (optional): Start bound for the scan range.
• options.end (optional): End bound for the scan range.
• options.prefix (optional): Key prefix to filter by. Only entries whose key starts with this prefix are returned.
• Returns: Array of ScanRow objects sorted by key in lexicographic order.
• Tombstones: Included in results with value: undefined. Use row.value === undefined to detect them.
• Empty result: Returns [] when no entries match.

Bound types:

// Inclusive: includes the boundary key
{ kind: "inclusive", key: ["b"] }

// Exclusive: excludes the boundary key
{ kind: "exclusive", key: ["d"] }

// Unbounded: no limit in this direction
{ kind: "unbounded" }

Examples:

// Full scan (all entries including tombstones)
const all = flock.scan();

// Prefix scan
const users = flock.scan({ prefix: ["users"] });

// Range scan with bounds
const range = flock.scan({
  start: { kind: "inclusive", key: ["b"] },
  end: { kind: "exclusive", key: ["d"] },
});

// Combine prefix and bounds
const page = flock.scan({
  prefix: ["users"],
  start: { kind: "exclusive", key: ["users", 100] },
  end: { kind: "inclusive", key: ["users", 200] },
});

ScanRow structure:

{
  key: KeyPart[];      // The full key
  raw: ExportRecord;   // Raw record with clock string and optional data/metadata
  value?: Value;       // The data value; undefined for tombstones
}

Replication and Import/Export

version(): VersionVector

Returns the exclusive (visible) version vector.

• Returns: VersionVector — only includes peers that currently own at least one visible entry (not overwritten by a later entry from another peer).
• Use case: Pass this to a remote peer as the from parameter of exportJson() to get only the updates the local replica is missing.
• Behavior after overwrites: If peer A's only entry is overwritten by peer B, peer A is removed from the exclusive version.

flock.setPeerId("a");
flock.put(["k"], "v1");
flock.version(); // { a: { physicalTime: ..., logicalCounter: ... } }

flock.setPeerId("b");
flock.put(["k"], "v2"); // overwrites peer a's entry
flock.version(); // { b: { ... } } — peer a is gone

inclusiveVersion(): VersionVector

Returns the inclusive (max-seen) version vector.

• Returns: VersionVector — tracks the highest clock seen from every peer during this instance's lifetime, including peers whose entries have been overwritten.
• Use case: Completeness checks, debugging, or determining whether a remote update has been "seen" even if it was subsequently overwritten.
• Guarantee: Always a superset of version().

flock.setPeerId("a");
flock.put(["k"], "v1");
flock.setPeerId("b");
flock.put(["k"], "v2");

flock.version();          // { b: { ... } }
flock.inclusiveVersion(); // { a: { ... }, b: { ... } }

merge(other: Flock): void

Merges all state from another Flock instance into this one.

• other: The source Flock instance.
• Effect: Applies all entries from other into this using LWW conflict resolution. After merge, this contains the union of both states.
• Idempotent: Calling merge(other) multiple times produces the same result as calling it once.
• Events: Emits an event batch for imported entries.
• Note: This is a one-directional merge. To fully sync two replicas, call a.merge(b) and b.merge(a).

exportJson(): ExportBundle

Exports the full state as a JSON bundle.

• Returns: ExportBundle containing all entries (including tombstones), keyed by stringified key arrays.
• Format: { version: number, entries: { [key: string]: ExportRecord } }.
• Use case: Full snapshot for backup, initial sync, or cold storage.

exportJson(from: VersionVector): ExportBundle

Exports only entries newer than the given version vector (incremental/delta export).

• from: The remote peer's current version vector (obtained via version()).
• Returns: Bundle containing only entries the remote peer hasn't seen.
• Use case: Bandwidth-efficient incremental sync. Send your version() to the remote, get back only what's new.

// Incremental sync pattern
const remoteVersion = remotePeer.version();
const delta = localPeer.exportJson(remoteVersion);
remotePeer.importJson(delta);

exportJson(from: VersionVector, pruneTombstonesBefore: number): ExportBundle

Exports with tombstone pruning.

• pruneTombstonesBefore: Millisecond timestamp. Tombstones with physicalTime older than this value are excluded from the export.
• Use case: Reduce export size by omitting old tombstones that all replicas have already seen.
• Caution: If a replica imports a pruned export, it may not know about deletions that happened before the prune threshold. Only use when you're confident all replicas have already processed those deletions.

exportJson(options: ExportOptions): Promise<ExportBundle>

Exports with hooks and/or additional options (async).

• options.from (optional): Version vector for incremental export.
• options.pruneTombstonesBefore (optional): Tombstone prune timestamp.
• options.peerId (optional): Export only entries written by this peer.
• options.hooks.transform (optional): Async function to transform each entry during export.
• Returns: Promise<ExportBundle>.
• Use case: Sign entries, redact sensitive data, or filter entries during export.

const signed = await flock.exportJson({
  hooks: {
    transform: async (ctx, payload) => {
      return {
        ...payload,
        metadata: { ...payload.metadata, sig: await sign(payload.data) },
      };
    },
  },
});

importJson(bundle: ExportBundle): void

Imports a JSON bundle, merging entries using LWW semantics.

• bundle: An ExportBundle from exportJson().
• Returns: void.
• Events: Emits event batches for newly applied entries.
• Effect on debounce: If autoDebounceCommit is active, any pending local events are flushed before the import events are delivered.

importJson(options: ImportOptions): Promise<void>

Imports with preprocessing hooks (async).

• options.bundle: The export bundle.
• options.hooks.preprocess: Async function (context, payload) => ImportDecision to validate each entry before import.
• Returns: Promise<void>.
• Use case: Validate signatures, check authorization, or reject untrusted entries.

await flock.importJson({
  bundle,
  hooks: {
    preprocess: async (ctx, payload) => {
      if (!await verify(payload.metadata?.sig, payload.data)) {
        return { accept: false, reason: "invalid signature" };
      }
      return { accept: true };
    },
  },
});

importJsonStr(bundle: string): void

Convenience method that parses a JSON string and imports it.

• bundle: JSON string representation of an ExportBundle.
• Returns: void.
• Use case: Import directly from a network response or file read without manual parsing.

kvToJson(): ExportBundle

Alias for exportJson() with no arguments. Returns the full state as a JSON bundle.

Hooks API

Hooks provide extension points for signing, validating, transforming, or filtering data during put, export, and import operations.

Put Hooks (PutWithMetaOptions)

type PutWithMetaOptions = {
  metadata?: MetadataMap;
  now?: number;
  hooks?: {
    transform?: (context: PutHookContext, payload: PutPayload) => MaybePromise<PutPayload | void>;
  };
};

• context: { key: KeyPart[], now?: number }.
• payload: { data?: Value, metadata?: MetadataMap } — a working copy.
• Behavior: The hook receives a cloned payload. It can mutate it in-place or return a new payload. If it returns void, the (possibly mutated) working copy is used.
• Constraint: The final payload must have a data field; omitting it throws TypeError.

Export Hooks (ExportOptions)

type ExportOptions = {
  from?: VersionVector;
  pruneTombstonesBefore?: number;
  peerId?: string;
  hooks?: {
    transform?: (context: ExportHookContext, payload: ExportPayload) => MaybePromise<ExportPayload | void>;
  };
};

• context: { key: KeyPart[], clock: EntryClock, raw: ExportRecord }.
• payload: { data?: Value, metadata?: MetadataMap } — a working copy.
• Behavior: Called once per entry in the export. Does not modify stored state; only affects the exported bundle.
• Use case: Encrypt data, redact PII, or append signatures before sharing.

Import Hooks (ImportOptions)

type ImportOptions = {
  bundle: ExportBundle;
  hooks?: {
    preprocess?: (context: ImportHookContext, payload: ImportPayload) => MaybePromise<ImportDecision>;
  };
};

• context: { key: KeyPart[], clock: EntryClock, raw: ExportRecord }.
• payload: { data?: Value, metadata?: MetadataMap } — a cloned copy for inspection (not mutation).
• ImportDecision: Return { accept: true }, { accept: false, reason: string }, or void (treated as accept).
• Behavior: Called once per entry before import. Rejected entries are excluded from the merge.
• Use case: Validate signatures, enforce ACLs, or reject entries from untrusted peers.

Events and Subscriptions

subscribe(listener: (batch: EventBatch) => void): () => void

Subscribes to change events.

• listener: Callback receiving EventBatch objects.
• Returns: An unsubscribe function. Call it to stop receiving events.
• Event delivery: Events are delivered synchronously after each state-changing operation (unless debounce or transactions are active).
• Re-entrancy safe: You can call put, delete, get, merge, or importJson from within a listener. New events from re-entrant operations are queued and delivered after the current batch completes.
• Error isolation: If a listener throws, other listeners still receive the event. The error does not propagate to the caller of the operation that triggered the event.
• Automatic cleanup: When all listeners are removed, the native WASM subscription is cleaned up.

EventBatch structure:

{
  source: string;   // "local" for put/delete/set, or peer identifier for import/merge
  events: Event[];  // Array of individual change events
}

Event structure:

{
  key: KeyPart[];          // The affected key
  value?: Value;           // New value; undefined for deletions
  metadata?: MetadataMap;  // Associated metadata
  payload: EventPayload;   // { data?: Value, metadata?: MetadataMap }
}

const unsub = flock.subscribe((batch) => {
  for (const event of batch.events) {
    console.log(`${batch.source}: ${event.key} → ${event.value}`);
  }
});

flock.put(["x"], 1); // listener fires: source="local", key=["x"], value=1
unsub(); // stop listening

Debounced Event Batching

Debouncing accumulates local events and delivers them as a single batch after a period of inactivity, reducing event noise during rapid-fire edits.

autoDebounceCommit(timeout: number, options?: { maxDebounceTime?: number }): void

Enables debounced event batching.

• timeout: Debounce window in milliseconds. Events are held until timeout ms of inactivity pass.
• options.maxDebounceTime (default: 10000): Maximum time in ms that events can be held, even if operations keep coming. This prevents unbounded delays during sustained writes.
• Effect: Local put/delete/set events are buffered instead of immediately delivered. Import and merge events are NOT debounced — they flush pending local events first, then deliver immediately.
• Throws: If a transaction is currently active, or if debounce is already enabled.

flock.autoDebounceCommit(100, { maxDebounceTime: 5000 });
flock.put(["a"], 1); // buffered
flock.put(["b"], 2); // buffered, timer reset
// After 100ms of inactivity → single batch with both events
// OR after 5000ms even if operations keep coming

commit(): void

Forces immediate emission of any pending debounced events.

• Effect: Flushes the debounce buffer and delivers all pending events as a batch.
• Does NOT disable debounce mode — subsequent operations continue to be debounced.
• No-op: If debounce is not active or no events are pending.

disableAutoDebounceCommit(): void

Disables debounce mode and flushes any pending events.

• Effect: Pending events are delivered immediately. Future operations emit events synchronously.
• No-op: If debounce is not active.

isAutoDebounceActive(): boolean

Returns whether debounce mode is currently enabled.

Transactions

Transactions batch multiple operations into a single atomic event delivery. All put/delete operations inside a transaction are delivered as one EventBatch on commit.

txn<T>(callback: () => T): T

Executes a synchronous callback within a transaction.

• callback: Synchronous function containing put/delete operations.
• Returns: The return value of the callback.
• Event batching: All events from operations inside the callback are collected and delivered as a single EventBatch when the transaction commits.
• Error handling: If the callback throws, the transaction is rolled back (event emission is canceled) and the error re-thrown. Important caveat: Data changes (puts/deletes) may NOT be rolled back — only event emission is affected. The underlying writes may have already been applied to the store.
• Throws: Error if called while autoDebounceCommit is active (mutual exclusion).
• Throws: Error if nested transactions are attempted.
• Constraint: The callback must be synchronous. Async operations inside txn() are not supported.

flock.subscribe((batch) => {
  console.log(`Received ${batch.events.length} events`);
});

flock.txn(() => {
  flock.put(["a"], 1);
  flock.put(["b"], 2);
  flock.put(["c"], 3);
});
// Listener receives: "Received 3 events" — single batch

isInTxn(): boolean

Returns whether a transaction is currently active.

Binary File Export/Import

The binary file format is Flock's most efficient serialization. Unlike JSON export (which requires parsing every entry and re-inserting them one by one into a B+Tree), the binary format stores pre-built B+Tree pages directly. This makes loading from binary dramatically faster — especially for large datasets.

| Operation | fromJson (JSON bundle) | fromFile (binary) | |-----------|--------------------------|---------------------| | Open cost | O(n log n) — parse JSON, encode each key, insert into B+Tree one by one | O(1) — read file header, done | | Size | Larger (JSON text, repeated key strings, base-10 numbers) | Smaller (binary-encoded keys, prefix-compressed leaves, compact integers) | | Use case | Network sync, incremental delta exchange | Persistence, checkpoints, full-state transfer |

Rule of thumb: Use exportJson/importJson for replication and sync. Use exportFile/fromFile for persistence and fast cold-start loading.

exportFile(): Uint8Array

Exports the entire Flock state as a binary file.

• Returns: Uint8Array containing the complete state in Flock's B+Tree page format.
• Format: Binary format with FLOK magic header, B+Tree pages with prefix-compressed keys, and optional WAL data. Significantly more compact than JSON export.
• Performance: The exported bytes contain ready-to-use B+Tree pages. When loaded via Flock.fromFile(), the pages are used directly without any re-indexing or re-encoding — opening is O(1) regardless of data size.
• Use case: Persist state to disk, transfer as a file, create compact backups, or achieve the fastest possible cold-start.
• Side effect: Flushes any pending memtable writes before exporting.

const bytes = flock.exportFile();
fs.writeFileSync("state.flock", bytes);

Restoration is done via Flock.fromFile():

const bytes = fs.readFileSync("state.flock");
const flock = Flock.fromFile(new Uint8Array(bytes), "my-peer");
// Ready instantly — no re-parsing or re-indexing needed

Type Reference

Version Vector Types

type VersionVectorEntry = {
  physicalTime: number;    // Wall-clock timestamp in milliseconds
  logicalCounter: number;  // Monotonic tie-breaking counter
};

interface VersionVector {
  [peer: string]: VersionVectorEntry | undefined;
}

Data Types

// Any JSON-serializable value
type Value = string | number | boolean | null | Array<Value> | { [key: string]: Value };

// A single part of a compound key
type KeyPart = Value;

// Arbitrary metadata stored alongside an entry
type MetadataMap = Record<string, unknown>;

Entry Types

type EntryClock = {
  physicalTime: number;    // When the entry was written
  logicalCounter: number;  // Tie-breaking counter
  peerId: string;          // Who wrote the entry
};

type EntryInfo = {
  data?: Value;            // Omitted for tombstones
  metadata: MetadataMap;   // Always present (defaults to {})
  clock: EntryClock;       // Clock of the winning write
};

Export/Import Types

type ExportRecord = {
  c: string;               // Clock string: "physicalTime,logicalCounter,peerId"
  d?: Value;               // Data value (omitted for tombstones)
  m?: MetadataMap;         // Metadata (omitted if empty)
};

type ExportBundle = {
  version: number;         // Format version number
  entries: Record<string, ExportRecord>;  // Key (stringified) → record
};

Hook Types

type ExportPayload = { data?: Value; metadata?: MetadataMap };
type ExportHookContext = { key: KeyPart[]; clock: EntryClock; raw: ExportRecord };
type ExportHooks = {
  transform?: (context: ExportHookContext, payload: ExportPayload) => MaybePromise<ExportPayload | void>;
};

type ImportPayload = ExportPayload;
type ImportHookContext = ExportHookContext;
type ImportAccept = { accept: true };
type ImportSkip = { accept: false; reason: string };
type ImportDecision = ImportAccept | ImportSkip | ImportPayload | void;
type ImportHooks = {
  preprocess?: (context: ImportHookContext, payload: ImportPayload) => MaybePromise<ImportDecision>;
};

type PutPayload = ExportPayload;
type PutHookContext = { key: KeyPart[]; now?: number };
type PutHooks = {
  transform?: (context: PutHookContext, payload: PutPayload) => MaybePromise<PutPayload | void>;
};

type PutWithMetaOptions = {
  metadata?: MetadataMap;
  now?: number;
  hooks?: PutHooks;
};

Scan Types

type ScanBound =
  | { kind: "inclusive"; key: KeyPart[] }
  | { kind: "exclusive"; key: KeyPart[] }
  | { kind: "unbounded" };

type ScanOptions = {
  start?: ScanBound;
  end?: ScanBound;
  prefix?: KeyPart[];
};

type ScanRow = {
  key: KeyPart[];
  raw: ExportRecord;
  value?: Value;
};

Event Types

type EventPayload = { data?: Value; metadata?: MetadataMap };

type Event = {
  key: KeyPart[];
  value?: Value;
  metadata?: MetadataMap;
  payload: EventPayload;
};

type EventBatch = {
  source: string;     // "local" or peer identifier
  events: Event[];
};

Conflict Resolution Deep Dive

How LWW Works in Practice

// Both peers start with the same state
const a = new Flock("peer-a");
const b = new Flock("peer-b");

// Concurrent writes at the same physical time
a.put(["key"], "alice-value", 1000);
b.put(["key"], "bob-value", 1000);

// After merge, both converge
a.merge(b);
b.merge(a);

// Who wins? Clock comparison:
// 1. physicalTime: both 1000 → tie
// 2. logicalCounter: both 0 → tie
// 3. peerId: "peer-b" > "peer-a" lexicographically → peer-b wins
console.log(a.get(["key"])); // "bob-value"
console.log(b.get(["key"])); // "bob-value"

Delete vs. Write Conflicts

// A writes, B deletes at a later time
a.put(["key"], "value", 1000);
b.delete(["key"], 2000);

a.merge(b);
// Delete wins because physicalTime 2000 > 1000
a.get(["key"]); // undefined

// A writes again at an even later time
a.put(["key"], "new-value", 3000);
a.get(["key"]); // "new-value" — write wins because 3000 > 2000

Version Vector After Overwrites

When a peer's entries are fully overwritten by another peer, the exclusive version vector drops the original peer:

const f = new Flock("a");
f.put(["k1"], "v1");     // version: { a: {...} }
f.setPeerId("b");
f.put(["k1"], "v2");     // version: { b: {...} } — a is dropped

// But inclusive version retains both:
f.inclusiveVersion();     // { a: {...}, b: {...} }

Version Vectors Explained

Incremental Sync Protocol

// Step 1: Bob sends his version to Alice
const bobVersion = bob.version();

// Step 2: Alice exports only what Bob hasn't seen
const delta = alice.exportJson(bobVersion);

// Step 3: Bob imports the delta
bob.importJson(delta);

// Step 4: Reverse direction
const aliceVersion = alice.version();
const reverseDelta = bob.exportJson(aliceVersion);
alice.importJson(reverseDelta);

// Both are now fully synced

When to Use Which Version

| Method | Includes overwritten peers | Use for | |---------------------|---------------------------|----------------------------------| | version() | No | Incremental sync, delta export | | inclusiveVersion()| Yes | Completeness checks, debugging |

Best Practices

1. Use Explicit Timestamps for Deterministic Testing

// In tests, always provide explicit timestamps for reproducible results
flock.put(["k"], "v", 1000);

// In production, omit the timestamp to use Date.now()
flock.put(["k"], "v");

2. Use Transactions for Batch Updates

// Bad: subscribers receive 100 separate event batches
for (let i = 0; i < 100; i++) {
  flock.put(["item", i], data[i]);
}

// Good: subscribers receive one batch with 100 events
flock.txn(() => {
  for (let i = 0; i < 100; i++) {
    flock.put(["item", i], data[i]);
  }
});

3. Use Debounce for Interactive Editing

// Accumulate rapid keystrokes into periodic batches
flock.autoDebounceCommit(200, { maxDebounceTime: 2000 });

// On each keystroke
flock.put(["doc", "content"], currentContent);

// Listeners receive updates at most every 200ms,
// or at least every 2000ms during sustained typing

4. Prune Tombstones Periodically

// When syncing, prune tombstones older than 7 days
const sevenDaysAgo = Date.now() - 7 * 24 * 60 * 60 * 1000;
const bundle = flock.exportJson(
  remoteVersion,
  sevenDaysAgo,
);

// Only safe when you know all replicas have synced within 7 days

5. Use Metadata for Audit Trails

flock.putWithMeta(["transaction", txId], amount, {
  metadata: {
    author: currentUser,
    sig: await sign(amount),
    timestamp: Date.now(),
  },
});

// Verify on import
const report = await flock.importJson({
  bundle,
  hooks: {
    preprocess: async (ctx, payload) => {
      if (!await verify(payload.metadata?.sig, payload.data)) {
        return { accept: false, reason: "signature verification failed" };
      }
    },
  },
});

6. Use Incremental Sync for Bandwidth Efficiency

// Don't do this (sends everything every time):
const full = flock.exportJson();
remotePeer.importJson(full);

// Do this instead:
const delta = flock.exportJson(remotePeer.version());
remotePeer.importJson(delta);

7. Design Keys for Scan Efficiency

// Good: hierarchical keys enable prefix scans
flock.put(["users", userId, "name"], "Alice");
flock.put(["users", userId, "email"], "[email protected]");
const userFields = flock.scan({ prefix: ["users", userId] });

// Bad: flat keys require full scans
flock.put(["user_name_" + userId], "Alice");
flock.put(["user_email_" + userId], "[email protected]");

Common Pitfalls

1. Debounce and Transactions Are Mutually Exclusive

flock.autoDebounceCommit(100);
flock.txn(() => { /* ... */ }); // Throws Error!

// Disable debounce first, or don't use both at the same time

2. Transaction Rollback Does NOT Undo Data Changes

flock.txn(() => {
  flock.put(["k"], "value");
  throw new Error("oops");
});
// Error is re-thrown, events are NOT emitted
// But flock.get(["k"]) may still return "value" — data persists

3. MVR Only Accepts Scalar Values

flock.putMvr(["k"], "string");  // OK
flock.putMvr(["k"], 42);        // OK
flock.putMvr(["k"], true);      // OK
flock.putMvr(["k"], null);      // Throws TypeError
flock.putMvr(["k"], [1, 2]);    // Throws TypeError
flock.putMvr(["k"], { a: 1 }); // Throws TypeError

4. merge() Is One-Directional

a.merge(b); // a now has everything from b
// But b does NOT have a's changes — you need:
b.merge(a);

5. Tombstones Appear in Scan Results

flock.put(["k"], "value");
flock.delete(["k"]);
const rows = flock.scan();
// rows includes { key: ["k"], value: undefined, raw: {...} }
// Filter tombstones if you only want live data:
const live = rows.filter(row => row.value !== undefined);

6. Import Flushes Debounced Events First

flock.autoDebounceCommit(5000);
flock.put(["local"], 1);       // Buffered, not emitted yet

flock.importJson(remoteDelta); // Forces flush of local batch FIRST,
                                // then delivers import batch
// Result: listener receives 2 batches in sequence

7. Peer ID Changes Don't Affect Past Entries

flock.setPeerId("a");
flock.put(["k"], "by-a");
flock.setPeerId("b");
// The entry at ["k"] is still attributed to peer "a"
// Only new operations will use peer "b"

Build

pnpm -C packages/flock-wasm build

Test

pnpm -C packages/flock-wasm test

License

@loro-dev/flock-wasm is licensed under AGPL-3.0-only.