@mintcd/sync-engine

Offline-first CRUD synchronization for Next.js applications backed by Cloudflare D1. Application mutations are written to IndexedDB and an operations queue first, then sent through generated API routes to D1. Remote operations can be pulled back into the browser through the generated service worker.

The package exports the client runtime. Its CLI is development-time code generation exposed by the same package through npx; application code never imports the CLI.

Install

Install the client runtime as an application dependency:

npm install @mintcd/sync-engine
npm install --save-dev wrangler

The core client runtime is the application dependency. Wrangler and the bundled generator CLI are used only during development/code generation.

The generator expects a Next.js project with a wrangler.jsonc in its root and a D1 binding named DB:

{
  "d1_databases": [
    {
      "binding": "DB",
      "database_name": "my_app",
      "database_id": "<database-id>"
    }
  ]
}

Create your application tables in D1 before generating code. The generator follows the selected Wrangler binding: "remote": true introspects hosted D1, while false or an omitted value introspects local D1. The schema must include an operations table. If it is missing, the interactive generator offers to create it in that selected database; migrations are preferable in non-interactive environments.

Configure and generate

Create a config file such as sync.config.ts:

import { defineSyncConfig } from '@mintcd/sync-engine/config';

export default defineSyncConfig({
  dbName: 'sync-engine-db',
  dbVersion: 1,
  outputPaths: {
    sw: '/public/sw.js',
    apiRoutes: '/app/api',
    engine: '/app/engine.ts',
  },
});

Run the generator from the project root:

npx @mintcd/sync-engine ./sync.config.ts

The config path can point to a .ts, .js, .mjs, .cjs, or .json file. The command reads the D1 schema through Wrangler and generates:

• CRUD and operations routes under outputPaths.apiRoutes;
• a service worker at outputPaths.sw;
• a typed db repository and finalConfig at outputPaths.engine.

To rebuild the remote operations log from the current contents of every application table, run:

npx @mintcd/sync-engine --bootstrap

Bootstrap does not load the sync config file. It reads wrangler.jsonc from the project root, always targets remote D1, creates the operations table if needed, clears existing operations, and writes one insert operation for every row in every other application table. An optional project root can be passed after the flag.

All output paths are relative to the project root, including paths beginning with /. Generated files are replaced on subsequent runs, so do not edit them manually. Regenerate after changing the D1 schema, and increment dbVersion whenever the IndexedDB store schema changes so existing browsers run an upgrade.

Use the client runtime

Initialize the service worker once in a client component, then use the generated repository:

'use client';

import {
  eq,
  syncNow,
  useLiveQuery,
  useSyncEngine,
  useSyncStatus,
} from '@mintcd/sync-engine';
import { db, finalConfig } from './engine';

export function Files() {
  useSyncEngine(finalConfig);

  const files = useLiveQuery(db.select().from('files'));
  const sync = useSyncStatus();

  async function createFile() {
    const now = Date.now();
    const result = await db.insert({
      name: 'Notes',
      created_at: now,
      updated_at: now,
    }).from('files').execute();
    return result.rows[0];
  }

  async function renameFile(id: string) {
    await db.update({ name: 'Renamed', updated_at: Date.now() })
      .from('files')
      .where(eq('id', id))
      .execute();
  }

  async function deleteFile(id: string) {
    await db.delete().from('files').where(eq('id', id)).execute();
  }

  return (
    <section>
      <p>Sync status: {sync.status}</p>
      <button onClick={createFile}>Create</button>
      <button onClick={() => void syncNow()}>Sync now</button>
      {files.loading && <p>Loading…</p>}
      {files.error && <p>{files.error}</p>}
      <pre>{JSON.stringify(files.data ?? [], null, 2)}</pre>
    </section>
  );
}

insert, update, and delete resolve after the local IndexedDB mutation is queued; network synchronization continues through the service worker. Mutation results include { affected, queued, opIds, rows }, and insert(...).execute() returns the locally inserted rows with their generated primary keys immediately. Generated repositories carry primary-key metadata, so TypeScript rejects a table's primary-key field in insert and update payloads; the service worker also rejects it at runtime. Use a where condition to select rows for updates. useLiveQuery refreshes when matching local data changes, useSyncStatus exposes online and synchronization state, and syncNow() requests an immediate synchronization pass.

Repository queries are immutable and carry a repository-scoped semantic key, so they can be created inline without useMemo. Equivalent live queries share their current result and execution while mounted. For query construction that should only run when explicit dependencies change, useLiveQuery also accepts a factory:

const file = useLiveQuery(
  () => db.select().from('files').where(eq('id', fileId)),
  [fileId],
);

Service workers require HTTPS in production; localhost is allowed for development.

Secure the generated routes

The generator cannot choose your application's authentication provider. Before deploying, protect the generated API route prefix with your Next.js middleware or route-level session checks. Do not expose the generated CRUD or operations endpoints anonymously. Service-worker requests are same-origin, so cookie-based application sessions can be enforced by those routes.

Development and release safety

Run the repository checks with:

npm test
npm run build
npm run test:e2e

The remote CRUD integration test is destructive: it deletes every row from statements, files, and operations in the remote D1 database before exercising CRUD. It is skipped unless both an opt-in flag and the exact disposable database name are supplied.

macOS/Linux:

SYNC_ENGINE_ALLOW_REMOTE_D1_RESET=1 \
SYNC_ENGINE_CONFIRM_TEST_DATABASE=sync_engine_db \
npm run test:e2e:integration

PowerShell:

$env:SYNC_ENGINE_ALLOW_REMOTE_D1_RESET = '1'
$env:SYNC_ENGINE_CONFIRM_TEST_DATABASE = 'sync_engine_db'
npm run test:e2e:integration

Never point that test at production or shared data. Keep npm and Cloudflare credentials out of source-controlled files. Publish npm explicitly from a verified local checkout with npm publish --access public; publishing a GitHub release runs verification only and never publishes the package again.

License

MIT