@fatagnus/dink-sync

Offline-first sync SDK for Dink edge platform with Effect.ts.

This SDK powers the Offline-First Edge type - edges that work offline and automatically sync when connectivity is restored.

| Edge Type | SDK | Use Case | |-----------|-----|----------| | Lite Edge | @fatagnus/dink-sdk | Always-connected IoT, RPC services | | Offline-First Edge | ✅ This SDK | Offline data with auto-sync | | Full Edge | Not yet available | Local NATS services (planned) |

Installation

npm install @fatagnus/dink-sync
# or
pnpm add @fatagnus/dink-sync

Features

• Offline-first: Queue changes locally and sync when online
• CRDT-based: Conflict-free replicated data types for automatic conflict resolution
• Real-time sync: Bidirectional sync with dinkd server via NATS
• React hooks: Ready-to-use hooks for React applications
• Persistence: Pluggable persistence providers (memory, PGlite)
• Type-safe: Full TypeScript support with strict typing
• Typed Client Factory: Generate type-safe collection access from Convex schema

Quick Start — ⭐ Typed Client (STRONGLY PREFERRED)

Always use typed clients generated from your Convex schema. This is the only recommended approach for production applications.

⚠️ Do not use the low-level sync engine or generic collection APIs in production. They should only be used for quick prototyping.

1. Define Your Schema

// convex/schema.ts
import { defineSchema, defineTable } from "convex/server";
import { v } from "convex/values";

export default defineSchema({
  tasks: defineTable({
    text: v.string(),
    isCompleted: v.boolean(),
    priority: v.optional(v.number()),
  }),
  users: defineTable({
    name: v.string(),
    email: v.string(),
  }),
});

2. Generate Typed Client

# Generate TypeScript types, validators, and typed client
dink codegen --convex-schema ./convex --convex-output ./src/generated --zod

3. Use the Typed Client

import { offlineEdge } from '@fatagnus/dink-sync';
import { PGlitePersistence } from '@fatagnus/dink-sync/persistence';
import { createTypedClient } from './generated';

// Initialize the offline edge client
const edge = offlineEdge.create({
  persistence: new PGlitePersistence(),
  config: {
    serverUrl: 'nats://localhost:4222',
    apiKey: 'your-edge-api-key',
  },
});
await edge.init();

// Create the typed client - this is the recommended way!
const db = createTypedClient(edge.get());

// Type-safe collection access with full IDE autocompletion
const task = await db.tasks.insert({
  text: 'Buy milk',
  isCompleted: false,
});
console.log(`Created task: ${task.id}`);

// List all tasks
const allTasks = await db.tasks.list();

// Update a task
const updated = await db.tasks.update(task.id, { isCompleted: true });

// Delete a task
await db.tasks.delete(task.id);

// Validation is automatic!
try {
  await db.tasks.insert({ text: '', isCompleted: false });
} catch (err) {
  console.error('Validation failed:', err.message); // "text is required"
}

Why Typed Client?

| Feature | Typed Client | Generic Collection | |---------|--------------|--------------------| | Type Safety | ✅ Compile-time checks | ⚠️ Runtime only | | IDE Support | ✅ Full autocompletion | ⚠️ Limited | | Validation | ✅ Automatic | ❌ Manual setup | | Refactoring | ✅ Safe renames | ⚠️ String-based |

Alternative: Low-Level Sync Engine ⚠️ Advanced/Prototyping Only

For advanced use cases or quick prototyping, you can use the sync engine directly. Migrate to typed clients before production:

import { createSyncEngine, NatsTransport } from '@fatagnus/dink-sync/client';

// Create transport and engine
const transport = new NatsTransport({
  serverUrl: 'nats://localhost:4222',
  apiKey: 'your-edge-api-key',
  appId: 'your-app-id',
  edgeId: 'your-edge-id',
});

const engine = createSyncEngine({
  serverUrl: 'nats://localhost:4222',
  apiKey: 'your-edge-api-key',
  transport,
});

// Connect and register documents
await engine.connect();
const actor = await engine.registerDocument('tasks', 'task-1');

// Queue changes
actor.queueChange(new Uint8Array([1, 2, 3]));

// Listen for sync events
engine.onSyncComplete((event) => {
  console.log(`Synced ${event.documentId}`);
});

React Integration

import { OfflineEdgeProvider, useCollection } from '@fatagnus/dink-sync/react';

function App() {
  return (
    <OfflineEdgeProvider config={{ serverUrl, apiKey, edgeId }}>
      <TaskList />
    </OfflineEdgeProvider>
  );
}

function TaskList() {
  const { items, insert, update, delete: remove } = useCollection('tasks');
  // ...
}

Testing

Unit Tests

Run unit tests (uses mock transport):

pnpm test

E2E Tests

E2E tests use testcontainers to spin up a real dinkd server.

Browser Tests

Browser tests verify PGlite persistence with IndexedDB in a real browser environment.

Prerequisites

1. Docker: Install and ensure Docker daemon is running

2. dinkd image: Build the dinkd Docker image from the project root:

   # From the project root directory
   docker build -t dinkd:latest .

Running E2E Tests

pnpm test:e2e

Running Browser Tests

Prerequisites:

1. Playwright browsers: Install Chromium for browser testing:

   npx playwright install chromium

Run browser tests:

pnpm test:browser

Browser test coverage:

• Data persistence with IndexedDB backend
• Data persistence across page reloads/sessions
• Large dataset handling (1000+ documents)
• Concurrent read/write operations
• Binary data integrity
• Special character handling in document IDs

E2E Test Coverage

The E2E tests verify:

• Edge connection to real dinkd server
• Document registration and sync operations
• External update subscription
• Concurrent sync from multiple edges
• Reconnection handling
• Error handling and event emission

Test Environment Variables

| Variable | Description | Default | |----------|-------------|--------| | DINK_TEST_IMAGE | Docker image for dinkd | dinkd:latest |

Troubleshooting Browser Tests

Playwright not installed:

• Run npx playwright install chromium to install the browser

Tests timing out:

• Large dataset tests may take longer; timeout is set to 60 seconds
• Consider running browser tests separately from unit tests

IndexedDB errors:

• Ensure you're running in a supported browser (Chromium)
• Check for quota limits in browser settings

Troubleshooting E2E Tests

Container startup timeout:

• Ensure Docker is running
• Check that the dinkd:latest image exists: docker images | grep dinkd
• Increase startup timeout in test configuration if needed

Connection refused:

• Container may not be fully started
• Check container logs for errors

Port conflicts:

• Testcontainers automatically maps to random ports, but conflicts can occur
• Stop any running dinkd containers manually if needed

API Reference

SyncEngine

• connect(): Connect to dinkd server
• disconnect(): Disconnect from server
• destroy(): Clean up all resources
• registerDocument(collection, docId): Register a document for sync
• unregisterDocument(collection, docId): Unregister a document
• getConnectionState(): Get current connection state
• onStateChange(callback): Subscribe to connection state changes
• onSyncStarted(callback): Subscribe to sync start events
• onSyncComplete(callback): Subscribe to sync completion events
• onSyncError(callback): Subscribe to sync error events
• onSyncRejected(callback): Subscribe to sync rejection events
• goOffline(): Manually force offline mode
• goOnline(): Exit manual offline mode
• discardLocalChanges(collection, docId): Discard pending changes for a document
• forcePush(collection, docId): Force push local state to server

DocumentActor

• queueChange(delta): Queue a change for sync
• hasPendingChanges(): Check if there are pending changes
• onPendingChange(callback): Subscribe to pending state changes
• getStateVector(): Get current state vector
• setStateVector(vector): Set state vector
• applyExternalUpdate(update): Apply an external update
• onExternalUpdate(callback): Subscribe to external updates

ConnectionState

• Offline: Not connected to server
• Connecting: Connection in progress
• Online: Connected and ready
• Reconnecting: Connection lost, attempting to reconnect

License

Apache-2.0