Frappe Live Sync SDK

Real-time sync for Frappe with TanStack DB integration, virtual pagination, active tab priority leadership, and enterprise features.

Documentation

docs/complete-reference.md

Comprehensive technical reference containing architecture, implementation details, server specification, and production deployment.

Live protocol spec (of record)

The wire protocol — change stream ordering, compound cursors, tombstones, idempotency, and Keycloak auth — is specified in agent-os/specs/2026-08-12-frappe-live-protocol/spec.md in the multiplatform.one repo. Where this README and the spec disagree, the spec wins.

Core Features

Virtual Pagination

• Progressive cache with page islands for efficient large datasets
• Cursor-based pagination via the Frappe Live extension app (live)
• Opt-in via progressive: true on subscription queries
• Automatic consistency across pages

Data Integrity

• Version checking with 409/412 errors (OCC via If-Match header)
• Three-way merge conflict resolution with field-level strategies
• Idempotency via Idempotency-Key headers — the server dedupes replayed create/delete requests (24h retention)
• CRDT-like field merging (sum, max, min, union-set, custom)

Offline Support

• IndexedDB / KV persistence via persistKey config
• Mutation journal with configurable TTL (default: 24hr)
• Automatic replay on reconnection
• Cross-tab coordination via BroadcastChannel + leader election

Real-time Sync

• WebSocket via Socket.IO with exponential backoff
• CDC with checkpoints for incremental sync after disconnects
• Tombstone support for tracking deletions
• Optional periodic CDC polling via cdcInterval for safety net
• Intelligent rehydration after mutations and remote changes

Advanced Features

• Active tab priority leadership with graceful handoffs
• Mutation batching (opt-in via enableBatching)
• Progressive pagination for infinite scroll with full sync
• Normalized cache with optional LRU eviction
• Devtools panel for inspecting subscriptions, cache, and journal

Installation

npm install @multiplatform.one/frappe

Authentication

auth.token accepts a string or an async getter — pass a getter so rotated tokens are resolved lazily per request (e.g. a Keycloak access token from @multiplatform.one/keycloak's getKeycloakBearerToken()):

import { AuthType, FrappeApp } from "@multiplatform.one/frappe";

const app = new FrappeApp("https://your-instance.frappe.cloud", {
  useToken: true,
  type: AuthType.Bearer,
  token: async () => (await getToken()) ?? "", // empty = guest, no bare header
});

The same Authorization header authenticates the realtime socket. Browsers cannot set headers on a WebSocket handshake, so on web the client handshakes over Socket.IO polling (the XHR carries the header) and upgrades to websocket afterwards; Node/native attach it to the websocket directly. On disconnect, connect_error, and every reconnect attempt the token getter is re-resolved into the live socket manager, so subscriptions and rooms survive token expiry mid-session (updateAuth() covers explicit rotation). With Keycloak, the bench validates the JWT via the frappe_keycloak app — REST and socket connections take the same validation path.

React Hooks

When to Use What

| Hook | Use Case | Updates | | ----------------------- | ---------------------------------------------------------------------------- | --------------------- | | useFrappeCollection | Creates a TanStack DB collection backed by a Frappe doctype | Auto-updates | | useLiveQuery | Reactive TanStack DB query with .where(), .orderBy(), etc. | Auto-updates | | useFrappeInfiniteList | Infinite scroll with full sync (realtime, offline, CDC) | Auto-updates | | useLinkSearch | Debounced backend link search (frappe.desk.search.search_link) for pickers | Per query (debounced) | | useLinkTitle | Resolve a link value's display title (get_link_title, session-cached) | On value change |

useFrappeCollection + useLiveQuery

import { useFrappeCollection, useLiveQuery, eq } from "@multiplatform.one/frappe";

// List with ordering
const collection = useFrappeCollection<Pokemon>(config, "Pokemon", { fields });
const { data } = useLiveQuery(
  (q) =>
    collection
      ? q.from({ pokemon: collection }).orderBy(({ pokemon }) => pokemon.poke_id, "asc")
      : null,
  [collection],
);

// Single document with filtering
const { data } = useLiveQuery(
  (q) =>
    collection
      ? q.from({ pokemon: collection }).where(({ pokemon }) => eq(pokemon.name, id))
      : null,
  [collection, id],
);

useFrappeInfiniteList

import { useFrappeInfiniteList } from "@multiplatform.one/frappe";

function PokemonList() {
  const { items, hasMore, isLoading, sentinelRef } = useFrappeInfiniteList<Pokemon>({
    config: { baseURL: "https://your-instance.frappe.cloud", auth: { token: "..." } },
    doctype: "Pokemon",
    fields: ["name", "pokemon_name", "poke_id"],
    orderBy: { field: "poke_id", order: "asc" },
    pageSize: 20,
  });

  if (isLoading && items.length === 0) return <Spinner />;

  return (
    <div>
      {items.map((pokemon) => (
        <PokemonCard key={pokemon.name} pokemon={pokemon} />
      ))}
      <div ref={sentinelRef}>{hasMore ? <Spinner /> : "No more items"}</div>
    </div>
  );
}

Quick Start

import { FrappeApp } from "@multiplatform.one/frappe";

const app = new FrappeApp("https://your-instance.frappe.cloud", {
  token: "your-token",
});

// Use the sync module for advanced features
const sync = app.sync();

// Subscribe to documents
const { subscriptionId } = await sync.subscribe({
  doctype: "Task",
  filters: { status: "Open" },
  limit: 20,
});

// Mutate with automatic conflict resolution
await sync.mutate({
  mutationId: crypto.randomUUID(),
  operation: "update",
  doc: { doctype: "Task", name: "TASK-001", status: "Complete" },
  version: task.modified,
});

Architecture

SyncModule (orchestrator)
├── SubscriptionManager
│   ├── Subscribe / unsubscribe lifecycle
│   ├── CDC backfill (backfillSub)
│   ├── Client-side filter evaluation
│   └── VirtualPageManager (opt-in progressive pagination)
│       ├── Page islands (contiguous + non-contiguous cache)
│       ├── Cursor + offset fetching
│       └── Remote change handlers
├── MutationEngine
│   ├── Optimistic create / update / delete with rollback
│   ├── Version checking (OCC via If-Match)
│   ├── Three-way merge with field-level strategies
│   ├── Cross-page coordination (global optimistic doc registry)
│   └── Optional batch queue (enableBatching)
├── MutationJournal
│   ├── Pending / confirmed / failed tracking
│   ├── TTL-based auto-cleanup
│   └── Rate-limited replay on reconnect
├── NormalizedCache
│   ├── Per-doctype document store
│   └── Optional LRU eviction (maxCacheSize)
├── RealtimeHandler
│   ├── Socket.IO doc_update / list_update events
│   ├── Authoritative REST fetch on event
│   └── Connection state management
├── RehydrationManager
│   ├── FetchItemOnly / FullRefresh strategies
│   ├── Priority-based task queue
│   └── Debounced processing
└── PersistenceManager
    ├── IndexedDB / KV snapshots (persistKey)
    ├── Multi-tab leader election (localStorage)
    ├── BroadcastChannel cross-tab sync
    └── Active tab priority handoff

Conflict Resolution

const sync = new SyncModule(db, realtime, {
  fieldMergeConfig: {
    Task: {
      tags: "union-set",
      progress: "max",
      time_spent: "sum",
      description: "server-wins",
      custom_field: (base, client, server) => {
        // Custom resolution logic
        return client ?? server;
      },
    },
  },
});

Server Requirements

The SDK works with standard Frappe REST API for reads and uses the Frappe Live extension app (live, endpoints live.live.api.*) for enhanced write operations. The extension provides:

• Idempotent create/delete — the server records each successful create_doc/delete_doc outcome against its Idempotency-Key (scoped per user and per endpoint, 24h TTL) and a retry with the same key returns the recorded outcome instead of re-executing. The SDK sends its client_mutation_id as this header on every mutation.
• OCC update via If-Match header (409/412 on version mismatch) — update retries are guarded by version checking, not idempotency dedup
• Cursor-based pagination via get_list_with_cursor
• Batch mutations via batch_mutations
• CDC endpoints via list_since and backfill — one merged change stream of rows + deletions, ordered by an opaque compound "timestamp|name" checkpoint that never advances past an undelivered change
• Tombstone feed — names-only deletions sourced from frappe core's Deleted Document rows (30-day retention; older cursors full-refresh) plus a doc_deleted realtime event after the deleting transaction commits
• Permission-checked reads — the change feed goes through frappe's permission engine; unauthorized doctypes raise instead of leaking rows

For Keycloak bearer auth, the bench additionally needs the frappe_keycloak app (JWT validation + role mapping on REST and socket connections).

Production Configuration

const sync = new SyncModule(db, realtime, {
  persistKey: "my-app", // Enable IndexedDB persistence
  cdcInterval: 30000, // Periodic CDC every 30s (safety net)
  backfillMaxPages: 20, // Max CDC pages drained per backfill (bounds both drain paths)
  maxCacheSize: 5000, // LRU eviction threshold
  mutationTTL: 24 * 60 * 60 * 1000, // 24hr mutation journal TTL
  enableBatching: false, // Opt-in mutation batching
  fieldMergeConfig: {
    /* ... */
  }, // Per-doctype merge strategies
  onConflict: (info) => console.warn("Conflict:", info),
  onMutationError: ({ mutation, err }) => reportError(err),
});

Testing

npm test

License

Apache 2.0