@y/yhub-http-fallback

HTTP provider for Yjs. Syncs a shared document over plain http requests instead of a websocket.

Every syncInterval milliseconds — and about debounce milliseconds after you stop typing — yhub-http-fallback performs a sync round: it PATCHes the local changes and GETs the whole remote document, which is merged into the local document. Merging is what Yjs is good at, so pulling the full document is safe and idempotent, no matter how long a client was away.

• Works wherever fetch works — no websocket upgrade, no long-lived connection
• Syncs awareness (presence), not just the document
• Speaks y/hub's rest api, so the same server serves your websocket clients and your http clients over the same rooms
• Designed to sit next to y-websocket as a fallback for clients that cannot open a socket

If you can run a websocket, run a websocket. yhub-http-fallback exists for the cases where you cannot: corporate proxies and captive portals that block upgrades, backends that only speak rest, and serverless deployments that have nowhere to keep a connection.

Caveats

Be honest with yourself about what polling can and cannot do.

• Remote changes arrive after up to syncInterval. Local changes go out about debounce milliseconds after the last one, so writes feel immediate — but you will not see someone else's keystrokes until the next round.
• Every round transfers the whole document. y/hub's read endpoint has no state-vector diff, so a 200 kB document polled every 10 seconds by 10 clients is a few megabytes of egress per minute. Raise syncInterval for large documents.
• Awareness lags, and expires. Awareness states that are not refreshed for 30 seconds are dropped, so syncInterval is capped at 15 seconds while awareness is enabled. Even then, cursor movement is only visible at poll resolution — good enough for "who is here", not for smooth remote carets.
• There is no push channel. The server cannot notify you, so there is no equivalent of y-websocket's connection-close. Permanent failures still surface — as a closed event on the round that hit them, rather than as a close frame.
• One poll loop per tab. yhub-http-fallback has no cross-tab BroadcastChannel support, so five open tabs mean five poll loops. y-websocket deduplicates tabs; yhub-http-fallback does not.

Quick start

npm i --save @y/yhub-http-fallback

import * as Y from 'yjs'
import { HttpProvider } from '@y/yhub-http-fallback'

const doc = new Y.Doc()
const provider = new HttpProvider(doc, 'https://my-hub.example.com/api', {
  org: 'my-org',
  docid: 'my-document-name'
})

provider.on('sync', () => {
  console.log('the document has been retrieved')
})

That is the whole configuration: the api endpoint, and the room that identifies the document. The same y/hub serves your websocket clients at wss://my-hub.example.com/api/ws/v1/my-org/my-document-name and your http clients at https://my-hub.example.com/api/ydoc/v1/my-org/my-document-name — same rooms, same document state, same authorization.

Authorization goes into params or headers, whichever your deployment reads. Both are plain objects on the provider and are read on every request, so refreshing a token is an assignment:

const provider = new HttpProvider(doc, 'https://my-hub.example.com/api', room, {
  params: { yauth: token } // ...or headers: { authorization: `Bearer ${token}` }
})

// `closed` fires when retrying would be pointless - see "Errors" below
provider.on('closed', async ({ code }) => {
  if (code === 401 || code === 403) {
    provider.params.yauth = await mintNewToken() // picked up by the next request
    provider.connect()                           // resume deliberately
  }
})

Errors

Every failed round is either transient — retrying may succeed — or permanent, meaning the request keeps failing until your application does something about it. yhub-http-fallback applies the rule that y/hub documents for its rest api, and that y-websocket applies to close codes:

| Status | Meaning | Retry? | |---|---|---| | 5xx | server-side failure — 500 internal error, 503 a dependency is temporarily down | yes, with backoff | | 429 | rate limited — Retry-After is honoured when present | yes, after the delay | | every other 4xx | 400 404 409 422 caller mistake, 401 403 unauthenticated / no access | no — fix the request or obtain fresh credentials | | anything that is not a response | a dropped connection, dns, a timeout | yes, with backoff |

A transient failure emits connection-error and the provider keeps polling, backing off exponentially up to maxBackoffTime.

A permanent failure emits connection-error and closed, and the provider stops: shouldConnect becomes false, no further requests are made, and unpublished changes are kept. It is not destroyed — act, then resume deliberately:

provider.on('closed', async ({ code, reason }) => {
  console.warn(`yhub-http-fallback gave up: ${code} ${reason}`)
  if (code === 401 || code === 403) {
    provider.headers.authorization = `Bearer ${await mintNewToken()}`
    provider.connect()
  }
})

shouldRetry implements the table above by default. Override it to opt out entirely, or to classify what your backend returns:

const provider = new HttpProvider(doc, serverUrl, room, {
  shouldRetry: () => true // never give up
})

It is deliberately written as a negation — an error that does not classify itself is transient — so a dropped connection never stops the provider by accident.

Use it as a y-websocket fallback

Some networks block websocket upgrades, and some deployments occasionally lose their websocket backend. yhub-http-fallback can stand in: both providers work on the same Y.Doc and the same Awareness instance, and a small helper makes sure only one of them is connected at a time.

Client

import * as Y from 'yjs'
import { Awareness } from 'y-protocols/awareness'
import { WebsocketProvider } from 'y-websocket'
import { HttpProvider, createWebsocketFallback } from '@y/yhub-http-fallback'

const doc = new Y.Doc()
// one Awareness instance, shared by both transports
const awareness = new Awareness(doc)

const wsProvider = new WebsocketProvider(
  'wss://my-hub.example.com/api/ws/v1/my-org', 'my-document-name', doc,
  { awareness }
)

const httpProvider = new HttpProvider(
  doc,
  'https://my-hub.example.com/api',
  { org: 'my-org', docid: 'my-document-name' },
  { awareness, connect: false } // the fallback helper decides when to poll
)

// poll over http from the moment the websocket is closed until it is established again
const stopFallback = createWebsocketFallback(wsProvider, httpProvider)

Three things matter here:

• The same Y.Doc. Nothing has to be flushed when the transport changes: whatever yhub-http-fallback retrieved while the websocket was down is part of the document, and the websocket's sync step publishes it as soon as the connection is back. Merging is what Yjs does.
• The same Awareness instance. Awareness state is keyed by doc.clientID, which belongs to the document. Two Awareness instances on one document would advertise the same client id with independent clocks and fight over the local state. Pass one instance to both providers so your cursor is published by whichever transport is connected, and so your editor binding sees remote cursors from both.
• connect: false. The helper owns the http provider's connection state. Don't call httpProvider.connect() yourself while it is installed.

Note how the same room is spelled differently by the two providers. y-websocket splits it into a server url and a room name, yhub-http-fallback takes it apart explicitly:

new WebsocketProvider(`wss://${host}/api/ws/v1/${org}`, docid, doc, { awareness })
new HttpProvider(doc, `https://${host}/api`, { org, docid }, { awareness })

What the helper does, exactly:

• While the websocket works, the http provider is disconnected — no requests at all.
• The websocket is closed → the http provider starts, on the first close. No grace period, no waiting for a second failure. This is the case the helper exists for: a network that kills websockets kills them every time, so waiting only delays the inevitable. The one exception is a close code that explicitly means "try again later" — see below.
• The websocket is established again → the http provider stops, and its in-flight request is aborted. This happens on the connected status, without waiting for the sync step to finish.
• Nothing is emitted at all → the http provider takes over after timeout. Some networks black-hole a connection instead of closing it, so no event ever arrives. The same timer covers a socket that connects but never completes its sync step.
• A provider that gave up is retried every retryInterval. After a permanent close y-websocket sets shouldConnect = false and never reconnects on its own, so without this the fallback would be a one-way trip — polling forever even after the problem was fixed. The same applies to the http provider when it fails permanently: it is retried, not abandoned.

Which closes start the fallback

shouldFallback decides. By default every close does, except the ones where the server explicitly said it will be back:

| Close code | | Fallback? | |---|---|---| | 1011 | internal error | no — wait out timeout | | 1013 | try again later | no — wait out timeout | | 4500-4599 | y/hub's transient range | no — wait out timeout | | 1006 | no close frame — a firewall, a proxy, a dropped link, a killed server | yes, immediately | | no code at all | a WebSocket polyfill that does not report one | yes, immediately | | everything else | 4400-4499 permanent, 1001 going away, … | yes, immediately |

Note which side of the line 1006 sits on. It is transient as far as reconnecting goes — that is why y-websocket keeps retrying it — but it is the exact signature of a network that does not allow websockets, and there is nothing to be gained by sitting on our hands while it retries. The two predicates answer different questions: shouldReconnect asks "should the socket try again?", shouldFallback asks "should we start polling in the meantime?".

1006 is also all a browser will ever tell you. CloseEvent.code is always a number — it is 1006 whenever no close frame arrived — and the reason behind a refused upgrade is deliberately hidden from javascript, so a 403 from a proxy appears only in the devtools console. That is why the default is written as a negation: anything that is not a recognised "try again later" starts the fallback, including a close that reports no code at all.

The cost of being eager is one http round trip on an ordinary blip that the websocket would have recovered from anyway. That is the trade: a few kilobytes against a user who cannot type. Override it if your network says otherwise:

createWebsocketFallback(wsProvider, httpProvider, {
  // only fall back when the websocket gave up entirely
  shouldFallback: event => event.code >= 4400 && event.code < 4500
})

A connection that you closed — wsProvider.disconnect() — never starts the fallback. A connection that y-websocket's watchdog closed because it stopped responding does, since that is a dead link by another name. Both arrive as a null event; shouldConnect is what tells them apart.

Nothing here takes the decision away from you. connect() on a provider that is already trying is a no-op, so the retry only revives one that stopped, and an application that handles closed itself — refreshing a token and calling connect() — recovers immediately instead of waiting for the next retry. Treat retryInterval as the safety net, not the mechanism.

The helper wants a reasonably recent y-websocket. Since [email protected] (and @y/websocket@4 for Yjs v14), a close code in the 4400-4499 range makes the provider stop reconnecting and emit closed; yhub-http-fallback emits the same event, with the http status as code. Both are the "retrying is pointless" signal described in Errors, and the helper listens to both.

It does not import y-websocket. The first argument needs synced, shouldConnect and connect(), the second shouldConnect, connect() and disconnect(); on()/off() on the first are optional. So it works with y-webrtc, with the hocuspocus/tiptap providers, or with your own — without events it falls back to re-checking every retryInterval, which costs nothing but a little latency when handing back.

One caveat worth stating plainly: a permanent websocket failure is often an auth failure, and http will hit the same wall. That is fine, and it is why the two halves fit together — yhub-http-fallback tries, gets its own permanent error, stops, and emits closed. Handle that event and you have one place to re-authenticate, whichever transport noticed first.

createWebsocketFallback returns a function that uninstalls it. It removes the listeners and clears the timer, and deliberately leaves both providers' connection state untouched:

stopFallback()

Server: y/hub

Nothing to do. The same host serves both endpoints over the same rooms:

• wss://{host}/api/ws/v1/{org}/{docid} — y-websocket
• https://{host}/api/ydoc/v1/{org}/{docid} — yhub-http-fallback

They share document state, awareness and authorization. A token that works as a query parameter or header on the websocket works on the rest endpoint too, and updates that a fallback client PATCHes are distributed to the connected websocket clients immediately — and the other way round.

Server: your own backend

yhub-http-fallback speaks two requests against one url, {serverUrl}/ydoc/v1/{org}/{docid}?branch=&gc=&awareness=. Bodies are lib0 any-encoded (buffer.encodeAny / buffer.decodeAny) — not json, not base64 — with content-type: application/octet-stream on requests.

| | | |---|---| | GET | Answer { doc: Uint8Array, awareness?: Uint8Array }. doc is the whole document, Y.encodeStateAsUpdate(doc). awareness is bare encodeAwarenessUpdate(...) output — no message type prefix — and is only asked for when awareness=true. | | PATCH | The body is { update?: Uint8Array, awareness?: Uint8Array }, at least one of them present. Apply what is there. The response body is ignored. | | errors | Any non-2xx status. A lib0-any { error: string } body is used as the message. The status decides whether the provider retries — see Errors. |

The query parameters are branch (default main), gc (default false) and awareness. Ignore the ones you don't implement.

Serving that from a y-websocket server, whose documents are already in memory behind getYDoc(docname), is a handful of lines on the same http.Server the websocket server is attached to:

import * as http from 'node:http'
import * as Y from 'yjs'
import * as buffer from 'lib0/buffer'
import { applyAwarenessUpdate, encodeAwarenessUpdate } from 'y-protocols/awareness'
import { getYDoc, setupWSConnection } from '@y/websocket-server/utils'
import { WebSocketServer } from 'ws'

const server = http.createServer(() => {})

server.on('request', async (req, res) => {
  const url = new URL(req.url, 'http://localhost')
  const match = /^\/api\/ydoc\/v1\/([^/]+)\/([^/]+)$/.exec(url.pathname)
  if (match == null) return
  // a y-websocket room is one string - map the {org}/{docid} pair onto it however you like
  const doc = getYDoc(`${match[1]}/${match[2]}`)
  const respond = body => {
    res.writeHead(200, { 'content-type': 'application/x-lib0any' })
    res.end(Buffer.from(buffer.encodeAny(body)))
  }

  if (req.method === 'PATCH') {
    const chunks = []
    for await (const chunk of req) chunks.push(chunk)
    const { update, awareness } = buffer.decodeAny(new Uint8Array(Buffer.concat(chunks)))
    // the same WSSharedDoc the websocket server uses, so this reaches every connected client
    if (update != null) Y.applyUpdate(doc, update, 'http')
    if (awareness != null) applyAwarenessUpdate(doc.awareness, awareness, 'http')
    respond({ success: true })
    return
  }

  const body = { doc: Y.encodeStateAsUpdate(doc) }
  if (url.searchParams.get('awareness') === 'true') {
    const clients = Array.from(doc.awareness.getStates().keys())
    if (clients.length > 0) body.awareness = encodeAwarenessUpdate(doc.awareness, clients)
  }
  respond(body)
})

const wss = new WebSocketServer({ noServer: true })
wss.on('connection', setupWSConnection)
server.on('upgrade', (req, socket, head) =>
  wss.handleUpgrade(req, socket, head, ws => wss.emit('connection', ws, req)))
server.listen(1234)

Point the client at it with new HttpProvider(doc, 'http://localhost:1234/api', { org, docid }).

Use @y/[email protected] for Yjs v13 — later versions target Yjs v14. The older y-websocket@2 shipped the same utilities at y-websocket/bin/utils; those are CommonJS, so a server file that mixes import * as Y from 'yjs' with require('y-websocket/bin/utils') ends up with two Yjs instances. Updates then fail Yjs' constructor checks and are silently dropped — awareness keeps working, which makes it look like a sync bug.

If your page is served from another origin, answer the OPTIONS preflight and send access-control-allow-origin / access-control-allow-headers — application/octet-stream is not a cors-simple content type.

What to expect during a failover

• A permanent failure on one transport usually means a permanent failure on the other. Listen to closed on both providers and re-authenticate in one place.
• Remote cursors flicker. y-websocket removes every remote awareness state when the socket drops; the next http round adds them back.
• Nothing is lost in either direction. Both providers publish from the same document, and Yjs merges.
• Latency changes, visibly. Say so in your ui if your users care — httpProvider.shouldConnect is the honest answer to "are we on the fallback right now".
• A brief blip costs an http round trip. The fallback starts on the first close and stops when the socket is back, so a two-second reconnect usually means one poll.
• The failover is not one-way. The websocket keeps being retried underneath, so a client that fell back during an outage returns to the socket on its own.

API

opts = {
  awareness: new Awareness(ydoc), // specify `null` to disable awareness
  connect: true,         // start syncing immediately

  syncInterval: 10000,   // sync at most this often, and never wait longer than this to publish
  debounce: 1000,        // publish local changes this long after the last one
  maxBackoffTime: 60000, // the longest delay between two failing rounds
  timeout: 30000,        // abort a round that takes longer than this. `0` disables it

  // Retrieve the garbage-collected document instead of the full history. Off by default: the
  // server's history features only work while the tombstones are still there.
  gc: false,

  // Decide whether a failed round is worth retrying. By default a `4xx` other than `429` is
  // permanent: the provider stops syncing and fires `closed`. See "Errors" above.
  shouldRetry: (error, provider) => error.retryable !== false,

  params: {},            // query parameters, read on every request
  headers: {},           // request headers, read on every request
  fetch: globalThis.fetch
}

syncInterval and debounce together describe when local changes are published: about debounce milliseconds after you stop typing, and — while you keep typing — at least every syncInterval milliseconds. A burst of changes results in a single request. syncInterval is also the polling interval, so it is how long a remote change may take to reach you.

gc selects which variant of the remote document is retrieved, and is deliberately not tied to doc.gc. It defaults to false, so the full history comes down: deleted content is only reconstructible while its tombstones are still around, which is what history, attribution and rollback features are built on. Set gc: true to retrieve the smaller garbage-collected variant instead. Either way your local document garbage-collects according to its own doc.gc.

There is no 'connection-close' event: http has no push channel, so there is nothing to report.

Recipes

Subdocuments. One provider per loaded subdocument, addressed by its guid:

doc.on('subdocs', ({ loaded }) => loaded.forEach(sub => {
  new HttpProvider(sub, serverUrl, { org, docid: sub.guid }, { awareness: null })
}))

Poll less often in a background tab.

document.addEventListener('visibilitychange', () => {
  provider.syncInterval = document.visibilityState === 'hidden' ? 60000 : 10000
})

Older runtimes. fetch and AbortController are expected to be global (node 18+). Pass opts.fetch to supply your own, or to route requests through a proxy agent.

Contributing

npm test        # runs an in-process fake y/hub and exercises a real client against it
npm run lint    # standard + tsc

There is one live test against a real y/hub, skipped unless you point it at one:

YHUB_URL=http://localhost:3002/api YHUB_ORG=testOrg npm test

License

The MIT License © Kevin Jahns