keepwell

Keep an app's documents, text or bytes, in the user's own Dropbox, Google Drive or OneDrive, in a shared SharePoint library, or in the Tiamat store, sealed so the server cannot read them. On a drive each document is one plain file, so the folder is also a backup the user can read, and a new device gets everything back by signing in. On a drive there is no server of ours: sign-in is OAuth with PKCE, straight from the app.

Install it with npm install @jojos-creations/keepwell. The plain npm name keepwell belongs to an unrelated package.

import { createSync, dropbox } from '@jojos-creations/keepwell';

const sync = createSync({ provider: dropbox({ clientId: DROPBOX_APP_KEY }), kv, redirectUri, readme: 'Do not delete.' });
location.assign(await sync.signInUrl());          // later, on the redirect: await sync.finishSignIn(location.href)
await sync.push({ name: 'workspace.json', text }); // a newer edit from another device comes back instead; the loser is kept as a copy
for (const doc of await sync.pull()) apply(doc);         // doc.deleted when another device removed it
await sync.remove('old.json');                     // gone on every device

• kv is any string store on the device, such as IndexedDB. keepwell keeps its tokens and last-seen revisions there.
• dropbox({ clientId }) uses the app's Dropbox App Folder, which the user sees under Apps. Register the Dropbox app with App Folder access, the files.content.read, files.content.write and files.metadata.read permissions, and your redirect URIs. Name it so the folder says not to delete it.
• googleDrive({ clientId, clientSecret }) uses Drive's appDataFolder, hidden from the user. Register an OAuth web client with the drive.appdata scope. Google requires its client secret even from public clients; it is not confidential.
• onedrive({ clientId }) uses the user's OneDrive app folder (Apps/<app name>), personal or work. Register an Entra app as a single-page app with your redirect URIs and the delegated Files.ReadWrite.AppFolder and offline_access permissions.
• onedrive({ clientId, library: { driveId, path } }) uses a folder in a SharePoint document library that several people share, such as a store's, with the delegated Sites.Selected permission: a tenant admin then grants the app write access to that one site (Graph's POST /sites/{site-id}/permissions), and each person's own access to the site still applies. Before its first write it checks that the tenant allows % in file names and that the folder's path is short enough, and says what to change if not. A shared library needs no account key: Microsoft 365 access is the boundary.
• tiamat({ clientId, folder }) keeps the account in the Tiamat store, signed in with the user's Tiamat account. Every name and body is sealed with the account key on the device, so the server cannot read them. Pass a keyStore; nothing syncs until the device has set up, recovered or paired the account key, and a full store rejects with quota. A store that hands back an older copy of a file than this device has already seen rejects with rolled-back, and nothing from it reaches the app; tell the user it is the server's doing.
• home({ folder, deviceName }) keeps the account on the user's own always-on computer, such as a desktop that holds their music. That computer runs createHomeHost over a folder (folderFiles from @jojos-creations/keepwell/node), listens on the home network with serveHomeOnLan, and, for devices away from home, keeps connectHomeRelay open to the Tiamat relay with the computer's Tiamat sign-in. A device pairs with a one-time invite from that computer through sync.signInWithCode(invite), not OAuth; show the invite as a QR code. Every request and answer is sealed end to end, so neither the home network nor the relay can read them. readRange reads part of a file, for streaming a track.
• memorySyncProvider() models an account in memory for tests.

A session

Most apps want the timing around those calls too. createSyncSession holds it, without touching the DOM:

const session = createSyncSession({
  kv,
  providers: { dropbox: () => dropbox({ clientId, folder: 'player' }) },
  sync: { redirectUri, readme: 'Do not delete.' },
  host: { localDocs: async () => myDocs(), apply: async (doc) => applyOrDelete(doc) },
});
await session.start();                              // before the app reads its documents
location.assign(await session.signInUrl('dropbox')); // later: await session.finishSignIn(location.href)
session.changed({ name: 'workspace.json', text });  // pushed after a quiet 1.5 s
addEventListener('visibilitychange', () => session.setForeground(document.visibilityState === 'visible'));

• A change is saved on the device at once and pushed after a quiet moment. It stays in get().unsent until a push succeeds, so a restart, stop() or an outage loses nothing; await session.changed(...) resolves once it is saved. Every pull pushes waiting changes first. A change the provider refuses (too large, say) stays waiting and is reported in error, without holding up other changes or pulls. Pulls run every minute and whenever the app calls pullNow. If apply throws, the documents it did not take come back on the next pull.
• start waits up to 4 s for the first pull, so an offline device still opens.
• setForeground(true) pulls at once, then again whenever the folder changes, so another device's change shows within seconds. Dropbox signals changes by long-poll and the Tiamat store over a WebSocket; Google Drive and OneDrive are polled every 15 s (foregroundPullEveryMs) instead. Behind, only the minute's pull runs. host.pulled runs after each pull, which is where an app re-reads its handoff states, and session.sync() is the Sync to make them with.
• applyLive: false holds changes pulled after startup in get().incoming until applyIncoming(), for an app whose running state would save over them. Held changes survive a restart and are applied by the next start.
• Joining an account keeps both sides: this device's localDocs are pushed first, so a document that differs from the account's is settled as a conflict, with the loser kept as a copy, and then everything is pulled.
• exclude(names) keeps some documents on this device only; removed on such a name deletes nothing remotely. get() also reports status, lastSyncedAt and the last error code, and subscribe tells the app when any of it changes.

Streams

For a log that every device adds to, such as history or an Automerge document's changes, a stream keeps each device's changes in its own files, so nothing conflicts:

await sync.appendChunks('history', [bytes]);              // one commit, however many chunks
const chunks = await sync.listChunks('history');          // every device's, snapshots included
const contents = await sync.readChunks(chunks);
await sync.writeSnapshot('history', merged, chunks);      // then deletes exactly those chunks

await sync.listStreams() and await sync.documents() name everything the folder holds, from every device.

A reader merges every chunk and snapshot it finds and must accept the same change twice, since two devices can snapshot at once. Pass { key } (32 random bytes) to every call of a stream to encrypt it.

Automerge documents

For app state that several devices edit at once, keep it in Automerge documents and give the Repo keepwell's storage adapter. Each device writes only its own chunks and Automerge merges them, so concurrent edits all survive and no clock decides anything:

import { Repo } from '@automerge/automerge-repo';
import { IndexedDBStorageAdapter } from '@automerge/automerge-repo-storage-indexeddb';
import * as Automerge from '@automerge/automerge';

const storage = automergeStorage(sync, { local: new IndexedDBStorageAdapter(), onError: showSyncError });
const repo = new Repo({ storage, network: [] });
// Every so often, and when the app comes back to the foreground:
for (const bytes of await storage.pull(handle.documentId)) handle.update((d) => Automerge.loadIncremental(d, bytes));

The device's own storage holds every chunk, so documents open offline; uploads that fail wait for the next save or pull. keepwell itself does not depend on Automerge.

Handoff

To pick up on one device where you left off on another, each device publishes its current state and reads the others'. Everything is encrypted under a key from the account key, so only paired devices can read it:

const tabs = handoff(sync, { name: 'tabs' });            // states expire after 7 days unless maxAgeMs says otherwise
await tabs.publish(bytes);                               // replaces this device's state
const others = await tabs.states();                      // every other device's latest, newest first
await tabs.send(phoneDeviceId, tabBytes);                // one item to one device
const items = await tabs.inbox();                        // what other devices sent here, oldest first
await tabs.done(items);
await tabs.prune();                                      // now and then: drops expired states and items

sync.deviceId() is this device's id; sync.devices() names the others. Call clear() when the user stops sharing.

The account key

One random key per account, shared by every app on it, for the app's own encryption (locked spaces). Pass a keyStore from the platform's secure store (SPEC.md has the table), never the kv:

const recovery = await sync.setUpAccountKey('Mac');       // first device: show the recovery code once
const { code } = await sync.requestPairing('iPad');       // new device: show as text and QR
await sync.approvePairing(typedOrScannedCode);            // a paired device
await sync.checkPairing();                                // new device, every few seconds: 'waiting' | 'paired'

await sync.devices() lists the account's devices for a devices screen; show an entry with signed: false as unconfirmed. removeDevice(id) takes one off the list but revokes nothing.

To remove a lost or sold device, revoke it. That replaces the account key and the vault key, hands the new ones to every other device this one already knew, and gives a new recovery code:

const result = await sync.revokeDevices([lostId], { pin: { pin, argon2id } }); // vault unlocked; pin: null drops the PIN
showRecoveryCodeOnce(result.recoveryCode);                                    // the old code opens nothing current
// result.unreached: devices this one could not hand the key to safely, which pair again

Revoking the device's sign-in is part of removing it. On the Tiamat store, revoke its sign-in at the account server in the same flow (POST /api/devices/revoke, which ends its OAuth refresh chain), so the server refuses everything it sends from then on. On Dropbox, Google Drive or OneDrive the documents themselves are plain files, so tell the user to sign that device out of the drive too.

Word it truthfully: the removed device keeps what it already has, but cannot read anything new and its changes are ignored by every device that took the new key. Without the sign-in revoked, that has limits a folder alone cannot close:

• A device that was offline throughout still trusts the removed one until it takes the new key, and can be handed a key the removed device made if that device writes a rotation of its own first: by deleting the real handover, or, for a device that missed two rotations, with nothing deleted at all, since the later rotation's cleanup already removed the handover that revoked it.
• Whoever can write the folder can delete it or put back an older account file. Devices refuse a rolled-back or missing account file rather than use their old key, but cannot bring the data back.
• A write under the old key while the rotation runs, between staging's copy of that file and the commit, is not carried over, since it may be the removed device's.
• A device entry the removed device planted, and this device already read, is reached. Show the list first so the user removes every device they do not recognize at once.
• A device left unreached can be impersonated, by a device that holds the new key and is stolen later, until it pairs again: flag unreached devices in the list until they do. A device that pairs again or recovers after being left unreached, or that missed a whole rotation, trusts what it first sees under the new key, as a newly paired device does. A device that missed a whole rotation also takes the new key with its vault locked: the PIN opens it, or without a PIN the new recovery code or pairing again, and its biometric unlock must be turned on again.
• Revoke a lost device before pairing its replacement. While the lost one is still paired it can sign another key into the replacement's entry; the approving device holds back for ten minutes, but other devices do not.

A key is only handed to a device whose key this device pinned before the revocation began; any other, and any entry whose key changed, is in unreached. A device that was removed reads keyStatus() as rotated. Other devices take the new key on their next call, with no code to type.

describeSyncError(error) gives one short sentence for any rejection, so apps word failures alike.

On the Tiamat store, apps keep a copy of the user's recovery code on the server by default, locked first with a synced passkey or with a PIN vault, so the server can never open it:

await sync.recoveryCopy?.lockWithPasskey(recovery, prfPasskeyLock(prf), credentialHint);
const copy = await sync.recoveryCopy?.read();                // new device, code lost
const code = await sync.recoveryCopy?.open(copy, { passkey: prfPasskeyLock(prf) }); // or { pin, vault }

Locked spaces

A space whose content needs the vault key, while its name shows on every paired device. Setup and recovery leave the vault unlocked, and so does pairing; a device whose vault locked again unlocks it with the account's PIN. Pass Argon2id from a library such as hash-wasm:

const argon2id = ({ password, salt, memoryKiB, iterations, parallelism }) =>
  hashWasm.argon2id({ password, salt, memorySize: memoryKiB, iterations, parallelism, hashLength: 32, outputType: 'binary' });
await sync.setVaultPin({ pin, argon2id });                           // with the vault unlocked
await sync.unlockVault(pin, { argon2id });                           // after a restart
await sync.enableBiometricUnlock(hardwareKey);                      // once, with the vault unlocked
await sync.unlockVaultWithBiometric(hardwareKey);                   // after a restart, no PIN
const keys = { accountKey: await sync.accountKey(), vaultKey: await sync.vaultKey() };
let space = await createLockedSpace({ ...keys, spaceId });
const sealed = await sealLockedSpace(space, { name, card: { panels: 3 }, body });
await sync.push({ name: `locked-${spaceId}`, bytes: sealed.bytes });
const card = await readLockedCard(doc.bytes, keys.accountKey);       // name and numbers, vault locked or not
const open = await unlockLockedSpace(doc.bytes, { ...keys, retired: await sync.retiredKeys() });
space = (await sealLockedSpace(open.space, { name, card, body }, keys)).space; // re-keyed if it was under a retired key

After a device is revoked, a space saved before then opens under a retired key with retired: true. Passing the current keys to sealLockedSpace re-keys it, which also gives it a new streamKey: write its streams under the new one from then on. Saving without them throws for a space under a retired key, including a handle made before the rotation.

Write the space's history and sessions to a stream with { key: card.streamKey }, or keep them on the device.

Several apps can share one registration and account: pass each its own folder, such as dropbox({ clientId, folder: 'player' }). See DESIGN.md for the conflict rules and SPEC.md for the folder format.

Developing

npm test, npm run typecheck, npm run lint, npm run build. An app can pin a commit over git; prepare builds dist/ on install.