@mintcd/sync-engine

A correctness-first TypeScript sync engine for offline-first applications.

The portable protocol, aggregate and record-oriented coordination, client coordinator, memory references, IndexedDB, HTTP and D1 persistence are implemented. Record mode uses individual indexed records and keyed protocol history. Vinext endpoint generation is also implemented, with explicit configuration and protection for application-owned files.

The accepted architecture, requirements, decisions, and implementation stages are in docs/. See the implemented API for the current contract.

Version 2.2 includes opt-in offline-first PWA integration: a worker sync host, foreground fallback, durable account bootstrap, and a Vinext/Vite plugin that builds and automatically registers an installable offline shell. The application supplies its offline UI, shared model and authentication. See the PWA setup and API, requirements, and verification. Version 2.2 also adds a schema-driven CRUD model and typed collection API: engine.from("notes").select("id", "title").where("folderId", "eq", "inbox").limit(50).execute(). The separate @mintcd/sync-engine-vinext package generates the sync route and one typed hook module from Wrangler and one TypeScript or JSON sync configuration. Import createSyncConfig from the package and useSyncEngine from the generated module for schema inference. Package helpers manage the engine, D1 authority and optional PWA wiring. Explicit discovery imports a reviewable D1 schema draft; existing-data adoption remains separate.

For Vinext applications, install the dedicated integration; it includes the core engine as a dependency:

npm install @mintcd/sync-engine-vinext

Use sync-engine.config.ts with defineSyncConfig from @mintcd/sync-engine-vinext/config for configuration suggestions. See the Vinext package guide for setup, generated files and the typed client API. Plain JSON configuration remains supported.

Core 2.3 adds bounded materialized authority checkpoints. Fresh record replicas can install a checkpoint at its canonical watermark, then continue incremental synchronization. Vinext applications can opt into a rebuild after each deployment with an explicit deploymentVersion; retained pending edits block replacement. See the bootstrap API and deployment setup.

Install and compose a browser client

npm install @mintcd/sync-engine

Version 2 introduces a new protocol and storage contract. Existing version 1 payloads and local databases require an application-managed migration; the package does not import legacy replicas automatically.

The package is ESM with TypeScript declarations and no runtime dependencies. Use a browser bundler for the client entrypoints. The application supplies a shared record model, a stable client ID, a stream ID, and an authorized sync endpoint. The model defines collection codecs, indexes, intents, operations, and optimistic/canonical reducers; see the record API and complete model example.

import { createSyncEngine } from "@mintcd/sync-engine/client";
import { openRecordReplica } from "@mintcd/sync-engine/records";
import { openIndexedDbRecordDatabase } from "@mintcd/sync-engine/adapters/indexeddb";
import { createHttpSyncTransport } from "@mintcd/sync-engine/transport/http-client";
import { model } from "./sync-model.js"; // Application-owned shared model.

export async function openBrowserSync(streamId: string, clientId: string) {
  const database = await openIndexedDbRecordDatabase({
    name: "my-app-sync", streamId, schema: model.schema,
  });
  try {
    const replica = await openRecordReplica({ database, clientId, model });
    const transport = createHttpSyncTransport({
      url: "/api/sync", streamId, model, credentials: "same-origin",
    });
    const engine = await createSyncEngine({
      replica, transport, autoSync: true,
      subscribeChanges: listener => database.subscribe(listener),
    });
    return {
      engine,
      async close() {
        try { await engine.close(); } finally { database.close(); }
      },
    };
  } catch (error) {
    database.close();
    throw error;
  }
}

Call engine.mutate(intent) to receive a durable local receipt, then engine.sync() to wait for a bounded exchange. Read/query visible records with engine.get() or engine.query() and inspect remote accept/reject outcomes with engine.readResolutions(). Reuse the same persisted clientId for that replica, including across tabs and reloads. Browser storage requires a browser context; create the client after hydration in an SSR application.

/api/sync and its authentication/origin/CSRF checks belong to the application. Configure transport headers when the endpoint needs tokens or additional headers. See the HTTP API for server composition and the vinext guide for endpoint generation. Automatic syncing performs bounded runs on startup and new local work; call sync() again when connectivity returns.

API documentation and runnable examples are included in the installed package under node_modules/@mintcd/sync-engine/docs/ and examples/. Run the record example without a database or endpoint:

node node_modules/@mintcd/sync-engine/examples/memory-records.mjs

Current package entrypoints

• @mintcd/sync-engine: portable types, typed errors, JSON codecs, intent fingerprints, and pure replica transitions.
• @mintcd/sync-engine/server: createSyncAuthority over an explicit store, application decision policy, and domain model; createRecordSyncAuthority for the record transaction capability.
• @mintcd/sync-engine/records: collection schemas, typed indexed queries, declared domain views, and openRecordReplica.
• @mintcd/sync-engine/client: createSyncEngine, subscriptions, bounded sync, cancellation, and typed transport failures.
• @mintcd/sync-engine/adapters/indexeddb: durable record storage with native indexes, atomic multi-store commits, and cross-connection invalidation.
• @mintcd/sync-engine/adapters/memory: reference replica and authority stores.
• @mintcd/sync-engine/adapters/d1: explicit migration generation and guarded record authority storage with searchable SQL columns.
• @mintcd/sync-engine/transport/http-client: bounded Fetch transport.
• @mintcd/sync-engine/transport/http-server: authorized Fetch endpoint factory.
• @mintcd/sync-engine/integrations/vinext: pure configuration validation and route planning; /integrations/vinext/node adds filesystem generation.
• sync-engine generate vinext: CLI for route/client-config generation, dry-run and drift checks.

The engine persists intent before exchange, assigns canonical sequence only on acceptance, and retains accepted/rejected decisions for safe retries. A policy can transform an intent into a different committed operation. Replicas reconcile contiguous pages and preserve accepted overlays during catch-up.

The memory stores exercise these contracts within one process. They do not persist through process restart. No browser storage or network is opened on import. See the client and IndexedDB API for browser composition, resource ownership, retry settings, and the explicit schema-upgrade boundary. The HTTP API and D1 API show the complete client-to-authority composition and server migration requirements. See vinext generation for setup, commands and the tested framework scope.

Run examples from a repository checkout

npm ci
npm run build
node examples/memory-counter.mjs
node examples/memory-records.mjs

The example saves an optimistic value of 8, then the authority accepts a transformed increment that confirms 5. The example source shows application composition without a framework or database dependency. The record example shows optimistic indexed queries, transformed acceptance, and rejection restoring the visible view. See the record API for composition and storage contracts.

Repository layout

• src/ contains hand-written production source, published for source navigation.
• test/ contains tests and is not published.
• docs/ contains published architecture and API documentation.
• examples/ contains published runnable local examples.
• dist/ contains generated, publishable package code.
• artifacts/ contains locally packed npm archives.

Development

npm ci
npm run check
npm run build
npm run test:browser
npm run test:vinext
npm run test:pwa

npm run check checks source, test, and consumer types, builds the package, and runs protocol, adapter, concurrency, and generated convergence scenarios. Runtime tests and shared fixtures are TypeScript. npm run build:tests compiles them into .tmp/test/; npm test runs the compiled tests with Node's test runner and source maps. No TypeScript loader is required. After building, run a focused case with node scripts/run-tests.mjs --test-name-pattern="converge". Checks also run a local Worker/D1 fixture; no remote database is needed. The separate browser command requires Chromium (or SYNC_ENGINE_BROWSER_PATH); setup is in the client API. Runtime dependencies are currently unnecessary. The vinext integration check uses pinned development dependencies and requires Node 22+; the portable core runtime and legacy generator retain the Node 20+ baseline. The dedicated Vinext package follows its Vite tooling Node range; see the package guide.