jsync-engine

v0.7.8

Published

17 days ago

Smart Sync Engine — offline-first realtime sync for local databases. CRDT conflict resolution, delta sync, guaranteed delivery.

Downloads

4,404

JSync Engine

Offline-first, realtime sync engine for local databases

Production-ready TypeScript SDK for building distributed applications with automatic conflict resolution, delta sync, and end-to-end encryption. Works seamlessly across web, mobile, and desktop platforms.

Full Documentation • Contributing • Support this project

What is JSync?

JSync is a distributed sync engine that makes offline-first applications effortless. Purpose-built for app data synchronization with the simplicity of Git combined with the real-time capabilities of Firebase.

When your users work offline, JSync automatically handles:

Tracking changes locally through delta tracking
Queuing mutations when disconnected
Resolving conflicts intelligently using CRDT algorithms
Syncing only changed fields (60-80% bandwidth reduction)
Encrypting all data end-to-end with AES-256-GCM
Maintaining causal ordering across devices with Vector Clocks
Guaranteeing eventual consistency across all devices
Providing immutable audit trails for compliance and debugging

Perfect for: task management, collaborative notes, offline-first mobile apps, real-time dashboards, and distributed team applications.

Why JSync?

Building distributed systems that sync reliably is complex. JSync eliminates the infrastructure challenges:

| Challenge | Solution | |-----------|----------| | Sync conflicts when devices disagree | CRDT algorithms with Vector Clocks and field-level merge strategies | | Bandwidth waste sending full documents | Delta sync transmits only changed fields (60-80% reduction) | | Data loss during poor connectivity | Persistent offline queue backed by SQLite with exponential backoff retry | | Network latency and disconnections | WebSocket with automatic reconnection handling | | Full database replication overhead | Partial replication syncs only required data per device | | Compliance and audit requirements | Immutable event log with complete change history | | Data security in transit and at rest | End-to-end encryption using AES-256-GCM | | Lack of type safety | Full TypeScript support with type inference |

Core Features

Offline-First Architecture

Application queries data instantly from local SQLite
All changes automatically sync when network connection is restored
Works seamlessly whether online or offline without code changes

React Hooks (NEW)

useSyncedQuery — live queries that auto-update on remote changes
useSyncState — read/write a single row, synced across devices
useSyncStatus — real-time connection and sync status

Admin Dashboard

Real-time monitoring of connected devices, latency, throughput
Conflict log viewer with strategy breakdown
API key management (generate, list, revoke)
Zero-dependency static HTML — no build step required

CRDT-Based Conflict Resolution

Last-Write-Wins strategy uses latest timestamp when devices conflict
Merge strategy resolves at field level for granular control
Custom resolvers available for complex business logic
Includes ORSet and LWW-Register data structures for advanced use cases

Delta Sync

Only changed fields are transmitted to server (not entire documents)
Reduces bandwidth consumption by 60-80% on average
Especially effective for large documents with frequent updates

Vector Clocks for Causal Ordering

Maintains proper event ordering across distributed devices
Ensures operations replay in correct causal sequence
No dependency on centralized timestamps

End-to-End Encryption

All data encrypted locally before transmission using AES-256-GCM
Encryption key management handled transparently
Server never has access to unencrypted application data

Guaranteed Delivery

Automatic retry mechanism with exponential backoff
Failed updates persist in local queue until successful sync
Total retry window approximately 5 minutes with configurable limits

Immutable Event Log

Complete audit trail of all changes stored permanently
Enables compliance reporting and data recovery
Supports time-travel debugging of data changes

Partial Replication

Mobile devices sync only their subset of data
Desktop applications sync complete datasets
Reduces storage and bandwidth requirements per device

Quick Start

For complete code examples and detailed tutorials, visit the full documentation.

Installation

npm install jsync-engine
pnpm add jsync-engine
yarn add jsync-engine

Basic Setup

import { createSyncedDB } from 'jsync-engine';

const db = await createSyncedDB({
  server: 'ws://localhost:3001/sync',
  database: 'myapp',
  apiKey: 'jsync_your-api-key-here', // Get from admin API
});

Server with Auth (Recommended)

import { SyncServer, AdminManager } from 'jsync-engine/server';

const admin = new AdminManager({ adminSecret: 'your-admin-secret' });
const server = new SyncServer({
  port: 3001,
  host: '0.0.0.0',
  auth: admin,
  adminSecret: 'your-admin-secret',
  conflictConfig: { defaultStrategy: 'merge' },
});

await server.start();

// Generate API keys via REST:
// POST /admin/keys { "userId": "[email protected]" }
// Header: x-admin-secret: your-admin-secret

Define Schema

await db.createTable({
  name: 'tasks',
  primaryKey: 'id',
  columns: [
    { name: 'id', type: 'TEXT' },
    { name: 'title', type: 'TEXT' },
    { name: 'status', type: 'TEXT' },
  ],
});

CRUD Operations

// Create
await db.insert('tasks', { id: '1', title: 'Build sync', status: 'in-progress' });

// Read
const tasks = await db.query('tasks', { status: 'in-progress' });

// Update
await db.update('tasks', '1', { status: 'completed' });

// Delete
await db.delete('tasks', '1');

Listen to Events

db.on('onStateChange', (state) => {
  console.log('Connected:', state.connected);
  console.log('Pending changes:', state.pendingCount);
});

db.on('onRemoteChange', (change) => {
  console.log('Remote update received:', change);
});

db.on('onConflict', (conflict) => {
  console.log('Conflict resolved:', conflict.resolution);
});

For step-by-step tutorials, complete examples, and advanced configurations, visit the documentation site.

Architecture

                    ┌─────────────────────────────────────┐
                    │          Your Application           │
                    │     (React, Vue, Node.js, etc.)     │
                    └──────────────────┬──────────────────┘
                                       │
                    ┌──────────────────▼──────────────────┐
                    │             JSync SDK               │
                    ├─────────────────────────────────────┤
                    │         • SyncedDatabase            │
                    │         • SyncClient                │
                    │         • ChangeTracker             │
                    │         • OfflineQueue              │
                    └──────────────────┬──────────────────┘
                                       │
                    ┌──────────────────▼──────────────────┐
                    │         Core Sync Engine            │
                    ├─────────────────────────────────────┤
                    │       • Delta Generator             │
                    │       • ConflictResolver (CRDT)     │
                    │       • VectorClock                 │
                    │       • Encryption (AES-256)        │
                    └──────────────────┬──────────────────┘
                                       │
                                  (WebSocket)
                                       │
                    ┌──────────────────▼──────────────────┐
                    │          JSync Sync Server          │
                    ├─────────────────────────────────────┤
                    │       • Event Log (immutable)       │
                    │       • Change Broadcaster          │
                    │       • Conflict Mediator           │
                    │       • Partial Replication         │
                    │       • Analytics Dashboard         │
                    └──────────────────┬──────────────────┘
                                       │
                    ┌──────────────────▼──────────────────┐
                    │         Persistent Storage          │
                    │     (MongoDB, PostgreSQL, etc.)     │
                    └─────────────────────────────────────┘

Performance

Query latency: <5ms (local SQLite)
Sync roundtrip: <100ms (sub-5ms with proper server location)
Bandwidth savings: 60-80% (delta sync)
Conflict resolution: <10ms (CRDT algorithms)
Encryption overhead: <5% (AES-256-GCM)

Security

End-to-End Encryption: AES-256-GCM
Message Authentication: HMAC-SHA256
API Key Authentication: Enforced on every WebSocket connection
Admin Key Management: REST API for generating, listing, and revoking keys
Secure Transport: HTTPS/WSS required
Secure Transport: HTTPS/WSS required
No Plaintext Logging: Sensitive data never logged or exposed

Supported Platforms

Node.js 18+
Browsers (Chrome, Firefox, Safari, Edge)
React Native (via @jsync-engine/react-native with expo-sqlite)
Electron
NW.js

Sync Server Setup

JSync requires a sync server to coordinate changes across devices. Two options are available:

Self-Hosted (Recommended for production)

Deploy your own sync server:

npm install @jsync-engine/server

See the documentation for deployment instructions.

Cloud Sync Service

Managed JSync service available (coming soon).

Troubleshooting

Offline changes not syncing

Check if the connection is active using the onStateChange event
Use forceSync() to manually trigger synchronization
Verify the server URL is correctly configured

Frequent conflicts

Review your conflict resolution strategy configuration
Consider switching from 'merge' to 'lww' if field-level conflicts persist
Examine application logic for concurrent writes to the same data

High memory usage

Increase the batch size to reduce queued change overhead
Reduce the number of local changes before syncing
Monitor the offline queue size in sync state

For additional help, refer to the documentation or open an issue on GitHub.

Examples

See the documentation for more examples:

Real-time todo app
Collaborative note-taking
Offline-first inventory management
Mobile app with sync

Benchmarks

Delta Sync Reduction

| Scenario | Without Delta | With Delta | Savings | |----------|---------------|-----------|---------| | Update 5 fields (100 fields total) | 2,191 B | 130 B | 94.1% | | Update single field (100 fields) | 2,191 B | ~50 B | 97%+ | | Batch 1000 changes | 500 KB | 50 KB | 90% |

Operations Per Second (single thread, Node.js)

| Operation | ops/sec | |-----------|---------| | Delta Generation (small) | 331,871 | | Conflict Resolution (LWW) | 943,370 | | Conflict Resolution (Merge) | 1,353,583 | | Vector Clock Increment | 4,714,979 | | Vector Clock Compare | 1,627,525 | | OR-Set Add | 705,756 | | PN-Counter Increment | 20,016,013 | | Encryption (AES-256-GCM) | 6,731 | | Decryption (AES-256-GCM) | 5,067 |

Run benchmarks yourself: pnpm bench

Contributing

We love contributions! See CONTRIBUTING.md for guidelines.

License

MIT — See LICENSE

Free and open source. No paid tiers, no usage limits.

Support

Documentation: jsync-engine.vercel.app
GitHub Issues: Report bugs and request features
Ko-fi: Support JSync development ☕

Acknowledgments

JSync draws inspiration from:

Git (version control, conflict resolution)
Firebase (realtime sync)
Redis (performance)
CRDTs (distributed systems)
Event Sourcing (audit trails)

Built by the JSync team.