wasm-memory-distributor

A JavaScript library for distributing WebAssembly memory over multiple machines using peer-to-peer synchronization.

Features

• Page-based memory tracking: Efficiently divides WASM memory into configurable pages (default 8KB)
• Automatic change detection: Periodically scans memory for changes using fast non-cryptographic hashing
• Binary diff/patch: Creates compact patches containing only changed bytes
• Conflict resolution: Uses vector clocks with Last-Write-Wins strategy for deterministic conflict resolution
• Zero dependencies: Pure JavaScript implementation
• Transport agnostic: Works with any transport layer (WebRTC, WebSocket, etc.)

Demos

• broadcast-channel-demo.html - Multi-tab demo using Broadcast Channel API
  • Live Demo on GitHub Pages

To run the demo locally:

npm install
npm run dev
# Open http://localhost:5173/broadcast-channel-demo.html

For the broadcast channel demo, open the URL in multiple browser tabs to see real-time synchronization across tabs.

Installation

npm install wasm-memory-distributor
# or
pnpm add wasm-memory-distributor
# or  
yarn add wasm-memory-distributor

Usage

import { 
  createMemoryDistributor, 
  scan, 
  applyPatches, 
  onDiff, 
  getVectorClock, 
  resizeMemory, 
  destroy 
} from 'wasm-memory-distributor';

// Create a WASM memory instance
const wasmMemory = new WebAssembly.Memory({ initial: 10 });

// Initialize the distributor
let distributorState = createMemoryDistributor(
  wasmMemory,
  'peer-id-123',
  {
    pageSize: 8192,      // 8KB pages (default)
    scanInterval: 100    // Scan every 100ms (default)
  }
);

// Listen for changes
distributorState = onDiff(distributorState, (patches) => {
  // Send patches to other peers via your transport layer
  sendToOtherPeers(patches);
});

// Manually trigger a scan
const scanResult = scan(distributorState);
distributorState = scanResult.state;
const patches = scanResult.patches;

// Apply patches received from other peers
distributorState = applyPatches(distributorState, receivedPatches);

// Get current vector clock state
const clock = getVectorClock(distributorState);

// Handle memory growth
wasmMemory.grow(5);
distributorState = resizeMemory(distributorState, wasmMemory.buffer);

// Clean up
distributorState = destroy(distributorState);

API

createMemoryDistributor(wasmMemory, peerId, config?)

Creates a new memory distributor state object.

• wasmMemory: WebAssembly.Memory instance
• peerId: Unique identifier for this peer (string)
• config (optional):
  • pageSize: Size of memory pages in bytes (default: 8192)
  • scanInterval: Milliseconds between automatic scans (default: 100, set to 0 to disable)

scan(state): { state, patches }

Manually triggers a memory scan and returns updated state and array of patches for modified pages.

onDiff(state, callback): state

Registers a callback to be invoked when changes are detected during automatic scans. Returns updated state.

applyPatches(state, patches): state

Applies received patches from other peers. Includes automatic conflict resolution using vector clocks. Returns updated state.

getVectorClock(state): VectorClock

Returns the current vector clock state for this peer.

resizeMemory(state, newBuffer): state

Updates the distributor when WASM memory grows. Pass the new buffer from the WebAssembly.Memory instance. Returns updated state.

destroy(state): state

Stops automatic scanning and cleans up resources. Returns updated state.

Patch Format

Each patch contains:

{
  pageIndex: number,           // Which page was modified
  patchData: Array<Change>,    // Binary delta changes
  vectorClock: VectorClock,    // Current vector clock state
  peerId: string               // Peer that made the change
}

Conflict Resolution

The library uses vector clocks to detect concurrent modifications. When conflicts occur:

1. Patches are compared using vector clocks
2. If clocks are concurrent (conflicting), Last-Write-Wins is applied
3. Peer IDs are compared lexicographically - higher ID wins
4. Both peers independently apply the same resolution logic to converge

License

ISC