npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2026 – Pkg Stats / Ryan Hefner

@edryslabs/genericprovider

v2.0.3

Published

A clean, minimal, backend-agnostic provider for Yjs that lets you implement any transport mechanism.

Readme

Generic Yjs Provider

A clean, minimal, backend-agnostic provider for Yjs that lets you implement any transport mechanism.

🚀 Live Demos

Try out all the different transport providers in action:

👉 View Live Demos

Explore 9 different transport implementations:

  • Dummy - In-memory testing with network simulation
  • SimplePeer - WebRTC peer-to-peer
  • PeerJS - Simplified WebRTC with cloud signaling
  • IndexedDB - Local browser persistence
  • Gun - Decentralized graph database
  • Trystero - Serverless WebRTC (Nostr, BitTorrent, MQTT, IPFS)
  • PubNub - Cloud pub/sub with global CDN
  • WebSocket - Standard client-server (y-websocket compatible)
  • Matrix - Federated communication protocol

Design Philosophy

You implement: How data is transmitted (WebSocket, WebRTC, PubNub, IndexedDB, etc.)
Provider handles: All Yjs synchronization logic (updates, awareness, state vectors)

Quick Start

1. Implement the Transport Interface

import type { Transport, ConnectionConfig } from './transport'

class MyTransport implements Transport {
  private connection: any
  private messageCallback?: (data: Uint8Array) => void
  
  get isConnected(): boolean {
    return !!this.connection
  }

  async connect(config: ConnectionConfig): Promise<void> {
    // Connect to your backend
    this.connection = await connectToBackend(config.room)
    
    // Listen for incoming data
    this.connection.on('data', (data: Uint8Array) => {
      if (this.messageCallback) {
        this.messageCallback(data)
      }
    })
  }

  disconnect(): void {
    this.connection?.close()
    this.connection = null
  }

  send(data: Uint8Array): void {
    this.connection?.send(data)
  }

  onMessage(callback: (data: Uint8Array) => void): () => void {
    this.messageCallback = callback
    return () => {
      this.messageCallback = undefined
    }
  }
}

2. Use the Provider

import * as Y from 'yjs'
import { GenericProvider } from './GenericProvider'
import { MyTransport } from './MyTransport'

// Create Yjs document
const doc = new Y.Doc()

// Create your transport
const transport = new MyTransport()

// Create provider
const provider = new GenericProvider(doc, transport)

// Connect
await provider.connect({
  room: 'my-room-123',
  url: 'wss://myserver.com'
})

// Use Yjs as normal - everything syncs automatically!
const ytext = doc.getText('content')
ytext.insert(0, 'Hello World')

// Access awareness for presence/cursors
provider.awareness.setLocalStateField('user', {
  name: 'Alice',
  color: '#ff0000'
})

// Listen to events
provider.on('synced', (synced) => {
  console.log('Synced:', synced)
})

provider.on('status', (status) => {
  console.log('Status:', status.state)
})

// Cleanup
provider.disconnect()
provider.destroy()

API Reference

Transport Interface

The minimal interface your transport must implement:

interface Transport {
  // Lifecycle
  connect(config: ConnectionConfig): Promise<void>
  disconnect(): void
  
  // Data transmission
  send(data: Uint8Array): void | Promise<void>
  onMessage(callback: (data: Uint8Array) => void): () => void
  
  // Status
  readonly isConnected: boolean
}

That's it! Just 4 methods + 1 property.

GenericProvider Class

class GenericProvider extends Observable<string> {
  // Properties
  readonly doc: Y.Doc
  readonly transport: Transport
  readonly awareness: Awareness
  readonly pubsub: PubSubChannel
  readonly status: ConnectionStatus
  readonly connected: boolean
  readonly synced: boolean

  // Methods
  constructor(doc: Y.Doc, transport: Transport, options?: {
    awareness?: Awareness
    syncInterval?: number     // Auto-sync interval in ms (default: 5000, set 0 to disable)
    verifyUpdates?: boolean   // Send hash with each update for fast desync detection (default: true)
    batchUpdates?: number     // Batch/debounce updates in ms (default: 0 = disabled, recommended: 50-200)
  })
  
  connect(config: ConnectionConfig): Promise<void>
  disconnect(): void
  destroy(): void
  syncNow(): void  // Force immediate sync (useful after network interruptions)
  
  // Events (inherited from Observable)
  on(event: 'status', callback: (status: ConnectionStatus) => void): void
  on(event: 'synced', callback: (synced: boolean) => void): void
  off(event: string, callback: Function): void
  emit(event: string, args: any[]): void
}

class PubSubChannel extends Observable<string> {
  // Subscribe to a topic
  subscribe(topic: string, callback: (message: any) => void): () => void
  
  // Publish to a topic
  publish(topic: string, message: any): void
}

Events

  • status: Emitted when connection status changes

    • { state: 'disconnected' }
    • { state: 'connecting' }
    • { state: 'connected' }
    • { state: 'error', error: Error }
  • synced: Emitted when initial sync completes

    • true when synced
    • false when connection lost

Built-in Features

✅ Automatic Document Sync

All changes to the Yjs document are automatically sent through your transport.

✅ Awareness Protocol

Presence information (cursors, users online, etc.) is handled automatically.

✅ State Vector Sync

Efficient synchronization using Yjs state vectors - only missing data is transmitted.

✅ Loop Prevention

Transaction origins prevent infinite update loops.

✅ Reconnection Ready

Can disconnect and reconnect without recreating the provider.

✅ Pub/Sub Channel

Built-in topic-based messaging for real-time communication that doesn't need CRDT properties (chat, notifications, RPC calls, etc).

Pub/Sub Channel

In addition to CRDT synchronization, the provider includes a pub/sub channel for real-time messaging:

// Subscribe to a topic
const unsubscribe = provider.pubsub.subscribe('chat', (message: any) => {
  console.log('Chat message:', message)
})

// Publish to a topic
provider.pubsub.publish('chat', {
  user: 'Alice',
  text: 'Hello!',
  timestamp: Date.now()
})

// Subscribe to all topics (wildcard)
provider.pubsub.subscribe('*', (message: any) => {
  console.log('All messages:', message)
})

// Unsubscribe when done
unsubscribe()

Use Cases for Pub/Sub

  • Chat Messages: Ephemeral messages that don't need to be stored in CRDT
  • Notifications: User joined, left, status changes
  • RPC Calls: Request/response patterns between clients
  • Events: Button clicks, interactions, temporary states
  • Presence: Real-time updates that complement awareness

Pub/Sub vs Awareness

  • Awareness: For user presence data (cursors, names, colors) - maintained by Yjs
  • Pub/Sub: For custom real-time messaging - you control the schema

Example: Chat System

// Setup
const doc = new Y.Doc()
const transport = new WebSocketTransport()
const provider = new GenericProvider(doc, transport)

await provider.connect({ room: 'chat-room' })

// Subscribe to chat
provider.pubsub.subscribe('chat', (msg) => {
  displayMessage(msg.user, msg.text)
})

// Send message
function sendMessage(text: string) {
  provider.pubsub.publish('chat', {
    user: getCurrentUser(),
    text: text,
    timestamp: Date.now()
  })
}

// The Yjs doc can still be used for persistent chat history if needed
const yhistory = doc.getArray('chatHistory')

Example Implementations

See examples.ts for complete implementations of:

  • WebSocketTransport: Real-time WebSocket communication
  • PubNubTransport: Pub/sub messaging
  • IndexedDBTransport: Local persistence (acts as a "transport")

How It Works

┌─────────────────────────────────────────────┐
│           Your Application                  │
│  (Uses Y.Doc, Y.Text, Y.Array, etc.)       │
└──────────────────┬──────────────────────────┘
                   │
                   │ Automatic sync
                   ▼
┌─────────────────────────────────────────────┐
│         GenericProvider                     │
│  • Listens to doc changes                  │
│  • Encodes updates with y-protocols        │
│  • Manages awareness                        │
│  • Handles sync protocol                    │
└──────────────────┬──────────────────────────┘
                   │
                   │ send(Uint8Array)
                   │ onMessage(callback)
                   ▼
┌─────────────────────────────────────────────┐
│         Your Transport                      │
│  • WebSocket.send()                         │
│  • PubNub.publish()                         │
│  • WebRTC.sendData()                        │
│  • IndexedDB.put()                          │
│  • etc.                                     │
└─────────────────────────────────────────────┘

Advanced Usage

Custom Awareness

import { Awareness } from 'y-protocols/awareness'

const awareness = new Awareness(doc)
const provider = new GenericProvider(doc, transport, { awareness })

Connection Configuration

Pass any config your transport needs:

await provider.connect({
  room: 'my-room',
  url: 'wss://server.com',
  apiKey: 'secret-key',
  timeout: 5000,
  // ... any custom fields
})

Async Transports

Your send() method can return a Promise:

async send(data: Uint8Array): Promise<void> {
  await this.backend.publish(data)
}

Error Handling

provider.on('status', (status) => {
  if (status.state === 'error') {
    console.error('Connection error:', status.error)
    // Implement retry logic
  }
})

Force Re-Sync

After network interruptions or when you suspect clients are out of sync, trigger an immediate sync:

// Manual re-sync
provider.syncNow()

// Re-sync when network comes back online
window.addEventListener('online', () => {
  console.log('Network restored')
  provider.syncNow()
})

// Re-sync after connection issues
provider.on('status', (status) => {
  if (status.state === 'connected') {
    // Give transport a moment to stabilize
    setTimeout(() => provider.syncNow(), 100)
  }
})

What syncNow() does:

  1. Sends the full local document state to all peers (ensures offline changes are transmitted)
  2. Sends a sync request (SyncStep1) to get missing updates from others
  3. Broadcasts the current awareness state

This is especially important after a client has been offline and made local changes, as it ensures those changes are sent to other clients when reconnecting.

Automatic Periodic Sync

By default, the provider automatically calls syncNow() every 5 seconds to recover from packet loss or intermittent network issues:

// Default behavior - auto-sync every 5 seconds
const provider = new GenericProvider(doc, transport)

// Faster recovery for unreliable networks
const provider = new GenericProvider(doc, transport, {
  syncInterval: 2000, // Retry every 2 seconds
})

// Disable automatic sync (not recommended)
const provider = new GenericProvider(doc, transport, {
  syncInterval: 0, // Manual sync only
})

Why automatic sync is important:

  • Yjs requires multi-step handshakes (SyncStep1 → SyncStep2 → Updates)
  • If any message is lost due to packet loss, sync stalls
  • Periodic retries ensure eventual consistency even on unreliable networks
  • Performance impact is minimal (only sends if there are changes)

For most production scenarios, the default 5-second interval provides good resilience without excessive traffic. For testing with simulated packet loss, use a shorter interval (e.g., 2 seconds).

Update Batching (Debouncing)

By default, every document change triggers an immediate network transmission. For performance optimization, you can enable update batching (also called debouncing):

// Default behavior - send updates immediately
const provider = new GenericProvider(doc, transport)

// Batch updates: collect changes and send after 100ms of inactivity
const provider = new GenericProvider(doc, transport, {
  batchUpdates: 100, // milliseconds
})

How it works:

  1. User types "hello" (5 keystrokes)
  2. Instead of 5 separate network messages, wait 100ms after last keystroke
  3. Send one combined update containing all changes
  4. Result: 1 message instead of 5

Benefits:

  • ✅ Reduces network traffic by ~70-90% during rapid typing
  • ✅ Lower bandwidth usage - fewer messages, less overhead per message
  • ✅ Better performance on slow or metered networks
  • ✅ Still feels instant - 50-200ms delay is imperceptible to users
  • ✅ Yjs automatically merges sequential updates efficiently

Recommended values:

  • 0 (default): Immediate transmission, no batching - best for slow networks or real-time cursors
  • 50-100ms: Good balance for most collaborative editing (typing, formatting)
  • 200ms: Maximum delay while still feeling responsive
  • 500ms+: Not recommended (users notice lag)

When to use:

  • ✅ Collaborative text editing (typing)
  • ✅ Drawing/whiteboard applications (mouse movements)
  • ✅ Metered or slow networks
  • ✅ Mobile/battery-conscious applications

When NOT to use:

  • ❌ Real-time cursor tracking (needs immediate updates)
  • ❌ Turn-based interactions (each change is discrete)
  • ❌ Ultra-low latency requirements (<50ms)

Example - Optimized collaborative editor:

const provider = new GenericProvider(doc, transport, {
  batchUpdates: 100,      // Batch updates for efficiency
  verifyUpdates: true,    // Hash verification for reliability
  syncInterval: 5000,     // Periodic backup sync
})

Fast Desync Detection

By default, the provider includes a document hash with every update for immediate detection of sync problems:

// Default behavior - hash verification enabled
const provider = new GenericProvider(doc, transport, {
  verifyUpdates: true, // Default: true
})

How it works:

  1. When sending an update, include hash of document state after the update
  2. Receiver applies update, computes own hash, compares
  3. If hashes don't match → immediately trigger re-sync (no waiting!)
  4. Much faster than waiting for periodic sync interval

Exponential backoff:

  • First mismatch: retry after 10ms
  • Second mismatch: retry after 50ms
  • Third mismatch: retry after 250ms
  • Fourth mismatch: retry after 1.25s
  • Further mismatches: retry after 6.25s, capped at 10s
  • If stable for 10 seconds, counter resets
  • Prevents "sync storm" if clients persistently disagree

Performance:

  • Hash computation is fast (~1ms for typical documents)
  • Only happens when updates occur (not constantly)
  • Adds minimal bandwidth (~4 bytes per update message)

When to disable:

// Disable for lowest possible bandwidth usage
const provider = new GenericProvider(doc, transport, {
  verifyUpdates: false, // Rely only on periodic sync
})

Only disable if you have a very reliable transport (no packet loss) or bandwidth is extremely constrained.

Comparison:

  • Without verification: Lost message → wait up to 5 seconds → retry
  • With verification (default): Lost message → next update detects mismatch → immediate retry

This makes sync much more responsive on unreliable networks!

Sequence Numbers for Ordering

When verifyUpdates is enabled, the provider automatically includes sequence numbers with each update for total ordering guarantees:

// Automatic with verifyUpdates enabled (default)
const provider = new GenericProvider(doc, transport, {
  verifyUpdates: true, // Includes sequence numbers
})

How it works:

  1. Each client maintains its own sequence counter (increments with each update sent)
  2. Updates include: [seqNum, clientID, update, hash]
  3. Receiver tracks last seen sequence number for each remote client
  4. Detects and handles:
    • Duplicates: seqNum ≤ last seen → skip (already processed)
    • Out-of-order: seqNum arrives before earlier ones → log warning
    • Gaps: seqNum jumps (e.g., 5 → 8) → log packet loss, Yjs handles recovery

Benefits:

  • Duplicate detection: Prevents processing the same update twice
  • Order verification: Confirms updates arrive in causal order
  • Gap detection: Identifies packet loss immediately (diagnostic)
  • Debugging: Easier to trace message flow and diagnose issues

Example console output:

[GenericProvider] Sequence gap detected from client 423891: expected 12, got 15 (gap of 3 messages)
[GenericProvider] Duplicate update detected from client 892301: seqNum 8 <= lastSeen 10

Performance:

  • Adds ~2-8 bytes per update (variable-length encoding)
  • O(1) duplicate detection (hash map lookup)
  • Negligible CPU overhead

Why this matters:

  • In unreliable networks (high packet loss), updates can arrive multiple times via retries
  • Without sequence numbers, duplicate updates would cause unnecessary reprocessing
  • With sequence numbers, duplicates are instantly detected and skipped
  • Provides diagnostic information about network quality

Built-in Rate Limiting

The provider includes automatic rate limiting on sync requests to prevent spam and protect against malicious or buggy clients:

// Default behavior - built-in rate limiting
const provider = new GenericProvider(doc, transport)
// Max 20 sync requests per 10-second window

How it works:

  1. Tracks all sync requests (SyncStep1) sent within a 10-second sliding window
  2. If more than 20 requests are attempted in that window, additional requests are dropped
  3. Warning logged to console when rate limit is hit
  4. Prevents "sync storm" scenarios where buggy code spams sync requests

Protection against:

  • Buggy code with infinite retry loops
  • Malicious clients attempting to flood the network
  • Accidental DoS from misconfigured exponential backoff
  • Network issues causing excessive retry attempts

Why this is safe:

  • 20 requests per 10 seconds = 2 per second average (very generous)
  • Normal operation uses far fewer requests (typically < 5 per 10 seconds)
  • Combined with exponential backoff, prevents most abuse scenarios
  • Doesn't affect normal document updates (only sync requests are limited)

The rate limiting is transparent and automatic - no configuration needed!

Migration from Existing Providers

If you have existing providers (like your PubNub/P2PT implementations), wrap them:

class PubNubTransportAdapter implements Transport {
  constructor(private oldProvider: OldPubNubSync) {}
  
  async connect(config) {
    return this.oldProvider.connect(config)
  }
  
  send(data: Uint8Array) {
    this.oldProvider.broadcast(true, data)
  }
  
  onMessage(callback) {
    this.oldProvider.onReceive = callback
    return () => { this.oldProvider.onReceive = undefined }
  }
  
  // ... etc
}

Optional Monitoring Utilities

The GenericProvider includes built-in hash verification (enabled by default with verifyUpdates: true). This is sufficient for most use cases.

However, for advanced diagnostics and monitoring, the SyncHealthMonitor utility is available:

import { SyncHealthMonitor } from './sync-monitor'

const monitor = new SyncHealthMonitor(provider, {
  checkInterval: 10000, // Check every 10 seconds
  onDesync: (details) => {
    // Send to error tracking, alerting system, etc.
    console.error('Desync detected across peers!', details)
  }
})
monitor.start()

When to use SyncHealthMonitor:

  • ✅ You need to monitor ALL peers simultaneously via pub/sub broadcasts
  • ✅ You want periodic health checks for diagnostics/alerting
  • ✅ You need separate monitoring infrastructure from core sync
  • ✅ Production monitoring and error tracking

When to use built-in verification (default):

  • ✅ Fast desync detection (immediate, on every update)
  • ✅ Zero configuration (automatic)
  • ✅ Exponential backoff, rate limiting, sequence numbers
  • ✅ Recommended for most applications

Key difference:

  • Built-in: Point-to-point verification (sender ↔ receiver) on each update
  • SyncHealthMonitor: Periodic broadcasts across all peers for diagnostics

Most applications should use the built-in verification. Use SyncHealthMonitor only if you need additional monitoring for alerting/diagnostics.

License

Same as LiaScript (BSD-3-Clause)