@wordup-md/unstorage-drivers

Isomorphic unstorage drivers, part of the WordUp ecosystem.

Every driver is its own subpath export: an app only loads the drivers it imports. Drivers rely on web standards (fetch, TextDecoder, URLSearchParams, the File System API), never on Node modules.

Install

pnpm add @wordup-md/unstorage-drivers unstorage

Drivers

| Driver | Import | Access | Runs in | | ---------- | --------------------------------------- | ---------- | --------------------- | | forgejo | @wordup-md/unstorage-drivers/forgejo | Read-only | Anywhere with fetch | | gitlab | @wordup-md/unstorage-drivers/gitlab | Read-only | Anywhere with fetch | | github | @wordup-md/unstorage-drivers/github | Read-only | Anywhere with fetch | | overlay | @wordup-md/unstorage-drivers/overlay | Read-write | Anywhere | | opfs | @wordup-md/unstorage-drivers/opfs | Read-write | Browser | | tauri-fs | @wordup-md/unstorage-drivers/tauri-fs | Read-write | Tauri app | | webdav | @wordup-md/unstorage-drivers/webdav | Read-write | Anywhere with fetch | | ipfs | @wordup-md/unstorage-drivers/ipfs | Read-write | Anywhere with Helia | | git-fs | @wordup-md/unstorage-drivers/git-fs | Read-write | Anywhere (adapter) |

forgejo and gitlab

Map the files of a Forgejo (or Gitea) or GitLab repository, the counterparts of unstorage/drivers/github. The git tree is listed once and kept for ttl seconds; file bodies are fetched on demand, pinned to branch.

import { createStorage } from "unstorage";
import forgejoDriver from "@wordup-md/unstorage-drivers/forgejo";
import gitlabDriver from "@wordup-md/unstorage-drivers/gitlab";

const docs = createStorage({
  driver: forgejoDriver({ apiURL: "https://codeberg.org", repo: "owner/name", dir: "content" }),
});
const site = createStorage({
  driver: gitlabDriver({
    repo: "group/name",
    getHeaders: async () => ({ Authorization: `Bearer ${await getToken()}` }),
  }),
});

await docs.getKeys(); // ["index.md", "docs:guide.md", ...]
await docs.getItem("index.md");
await docs.getKeys("", { force: true }); // refetch the tree now

| Option | Type | Default | Description | | ------------ | ------------------- | ----------------------------------------------------------------- | ---------------------------------------------------------------------- | | repo | string | required | owner/name of the repository (GitLab: project path or ID). | | branch | string | Forgejo: "main", GitLab: the project's default branch | Branch, tag or commit SHA to read. | | dir | string | "" | Directory of the repository used as the driver root. | | ttl | number | 600 | Seconds the file list is reused. Infinity for a pinned commit. | | token | string | | Access token, needed for private repositories. | | getHeaders | () => HeadersInit | | Headers resolved before every request (OAuth bearer, refreshed token). | | apiURL | string | Forgejo: "https://codeberg.org", GitLab: "https://gitlab.com" | Base URL of the instance. | | fetch | typeof fetch | globalThis.fetch | Used for every request, e.g. @tauri-apps/plugin-http to bypass CORS. |

getMeta(key) returns the blob sha and mode (and size on Forgejo).

github

The same driver for GitHub, with the options of forgejo: unstorage/drivers/github without its cdnURL, with getHeaders and an injectable fetch. The tree is listed in one request, or one per directory when GitHub truncates it (very large repositories); files are read raw from the contents API, which answers browsers with CORS headers.

import githubDriver from "@wordup-md/unstorage-drivers/github";

const docs = createStorage({
  driver: githubDriver({ repo: "owner/name", dir: "content", token: githubToken }),
});
// GitHub Enterprise Server:
githubDriver({ apiURL: "https://github.example.com/api/v3", repo: "owner/name" });

Options as in the table above, with branch defaulting to "main" and apiURL to "https://api.github.com". The token goes as Authorization: Bearer … (a fine-grained token with read access to the contents is enough). getMeta(key) returns the blob sha, mode and size. Without a token, GitHub allows 60 requests an hour per IP address: one for the tree, one per file read.

overlay

Stacks layers over a base, to edit a read-only source locally: layers[0] is the writable top layer, the last layer is the base. The top layer only ever holds the changes: a value set back to the base one is dropped, a removed base file leaves a marker. getChanges() lists them, ready for a commit.

import indexedDbDriver from "unstorage/drivers/indexedb";
import overlayDriver from "@wordup-md/unstorage-drivers/overlay";

const overlay = overlayDriver({
  layers: [indexedDbDriver({ dbName: "changes" }), gitlabDriver({ repo: "group/name" })],
});
const storage = createStorage({ driver: overlay });

await storage.setItem("index.md", "# Home, edited");
await storage.getItem("index.md", { original: true }); // the base value
// A rename: remove the old file, then write the new one with its `previousPath`.
await storage.removeItem("docs:guide.md");
await storage.setItem("docs:intro.md", "# Intro", { previousPath: "docs/guide.md" });

const changes = await overlay.getChanges();
// [
//   { type: "move", key: "docs:intro.md", previousKey: "docs:guide.md", value: "# Intro", originalSha: "…" },
//   { type: "update", key: "index.md", value: "# Home, edited", originalSha: "…" },
// ]

// Once the changes are committed to the base:
for (const { key } of changes) await overlay.mergeItem(key);
await storage.getKeys("", { force: true }); // relist the base now rather than after its `ttl`

getChanges() compares the top layer with the layers under it and returns one entry per changed file, sorted by key, leaving out the values back to the ones below:

| Field | Type | Description | | ------------- | -------------------------------------------- | ------------------------------------------------------------------------------------- | | type | "create" \| "update" \| "delete" \| "move" | move: renamed from previousKey, its value possibly edited too. | | key | string | Key of the file, the new one for a move. | | previousKey | string | Key the file was moved from. | | value | unknown | Value as the top layer holds it, read raw. None on a delete. | | originalSha | string | sha in the getMeta() of the original below (of previousKey for a move), if any. |

A move is a file written with a previousPath whose previous key is still below, hidden by a removal marker, to a key free below: it stands for both, so the marker is not listed as a delete. Renamed over a file below, it is an update and a delete; renamed from a key still there, a copy (create). mergeItem(key) drops the value and meta of key, and for a move the marker of its previous key.

opfs

Files in the Origin Private File System of the browser, or in any FileSystemDirectoryHandle (e.g. a folder picked with showDirectoryPicker()).

| Option | Type | Default | Description | | -------- | --------------------------- | ---------------------------------- | ---------------------------------------------------- | | fs | FileSystemDirectoryHandle | navigator.storage.getDirectory() | Directory used as the filesystem root. | | base | string | "" | Directory under the root used as the driver root. | | ignore | (path) => boolean | | Skips the files and directories it returns true for. |

getMeta(key) returns the mtime, size and type of the file.

tauri-fs

The files of a local directory in a Tauri app, through its fs plugin. The plugin is passed in rather than imported, so the driver loads anywhere and the app decides when to load the plugin.

import { open } from "@tauri-apps/plugin-dialog";
import tauriFsDriver from "@wordup-md/unstorage-drivers/tauri-fs";

const base = await open({ directory: true, recursive: true }); // adds the folder to the fs scope
const storage = createStorage({
  driver: tauriFsDriver({ fs: await import("@tauri-apps/plugin-fs"), base }),
});

await storage.setItem("docs:guide.md", "# Guide"); // writes <base>/docs/guide.md
const unwatch = await storage.watch((event, key) => console.log(event, key));

| Option | Type | Default | Description | | ------------ | ------------------- | ------------------------------- | ------------------------------------------------------------ | | fs | TauriFs | required | The fs plugin, await import("@tauri-apps/plugin-fs"). | | base | string | required | Driver root: an absolute path, or one relative to baseDir. | | baseDir | BaseDirectory | | Base directory base is relative to, e.g. AppData. | | ignore | (path) => boolean | skips .git and node_modules | Skips the files and directories it returns true for. | | watchDelay | number | 2000 | Milliseconds without change before the watcher reports. |

getMeta(key) returns the mtime, atime, birthtime and size of the file. Links to files are listed, links to directories are not. watch() reports the files created, changed, removed or renamed under base, including from other apps; it needs an absolute base, the watch feature of tauri-plugin-fs and the fs:allow-watch and fs:allow-unwatch permissions. The app must allow base in the fs scope: the dialog plugin adds the folders it opens, and the persisted-scope plugin keeps them across restarts. @tauri-apps/plugin-fs is an optional peer dependency: the driver only imports its types.

webdav

The files of a WebDAV collection: Nextcloud, OpenCloud, SFTPGo, FileBrowser… The file list is read with PROPFIND, one Depth: 1 request per directory (Nextcloud refuses infinity), and kept for ttl seconds; file bodies are read and written on demand, and a write creates the missing collections.

import { createAuth } from "@wordup-md/auth";
import { oidc } from "@wordup-md/auth/oidc";
import webdavDriver from "@wordup-md/unstorage-drivers/webdav";

const auth = createAuth(await oidc({ issuer: "https://auth.example.com", clientId: "wordup" }));
const storage = createStorage({
  driver: webdavDriver({
    url: "https://cloud.example.com/remote.php/dav/files/me/site",
    getHeaders: auth.getHeaders, // `Authorization: Bearer …`, OpenCloud or Nextcloud `user_oidc`
  }),
});

const { sha } = await storage.getMeta("content:index.md"); // the etag
await storage.setItem("content:index.md", "# Home", { ifMatch: sha }); // `conflict` if it changed

| Option | Type | Default | Description | | ------------ | ------------------- | ------------------------------- | --------------------------------------------------------------------------------------- | | url | string | required | URL of the collection used as the driver root. | | token | string | | Sent as Authorization: Bearer <token>. | | getHeaders | () => HeadersInit | | Headers of every request, e.g. a refreshed OAuth token, or Basic for an app password. | | ttl | number | 60 | Seconds the file list is reused. | | ignore | (path) => boolean | skips .git and node_modules | Skips the files and directories it returns true for. | | fetch | typeof fetch | globalThis.fetch | E.g. the one of @tauri-apps/plugin-http, without CORS. |

getMeta(key) returns the etag (also as sha, so the overlay reports it as the originalSha of a change), size, mtime and type. setItem, setItemRaw and removeItem take ifMatch (an etag) and ifNoneMatch: "*": a failed condition throws an error with code: "conflict". In a browser, the server must allow CORS for the app, and expose the ETag header of its PUT answers (without it, the list is read again after a write).

ipfs

A UnixFS directory on IPFS, with Helia. The whole storage is one directory DAG, so its state is a single root CID: every write builds a new root, from the changed file up, and hands it to onRootChange() to persist or publish. Pass it back as root to reopen the same storage.

import { createHelia } from "helia";
import { unixfs } from "@helia/unixfs";
import { CID } from "multiformats/cid";
import ipfsDriver from "@wordup-md/unstorage-drivers/ipfs";

const helia = await createHelia();
const saved = localStorage.getItem("root");
const driver = ipfsDriver({
  fs: unixfs(helia),
  root: saved ? CID.parse(saved) : undefined,
  onRootChange: (root) => localStorage.setItem("root", root.toString()),
});
const storage = createStorage({ driver });

await storage.setItem("docs:guide.md", "# Guide");
await driver.getRoot(); // the CID of the whole storage

| Option | Type | Default | Description | | -------------- | --------------------- | --------------------- | ------------------------------------------- | | fs | UnixFS | required | The UnixFS instance, e.g. unixfs(helia). | | root | CID | a new empty directory | CID of the directory to open. | | onRootChange | (root: CID) => void | | Called with the new root after every write. |

Writes run one at a time; a failed write keeps the previous root. Removing a file prunes the directories it leaves empty, so the same files always make the same root. getMeta(key) returns the file cid, size, mode and mtime. @helia/unixfs and multiformats are optional peer dependencies: the driver only imports their types.

git-fs

Not a driver but the other way around: a filesystem over any unstorage instance, for isomorphic-git. Files are storage items (path a/b.md is key a:b.md), directories are implicit, and reads and writes go raw so git objects stay byte-exact on any driver. A file written with storage.setItem() is a file git sees.

import git from "isomorphic-git";
import http from "isomorphic-git/http/web";
import { createStorage, prefixStorage } from "unstorage";
import createGitFs from "@wordup-md/unstorage-drivers/git-fs";
import opfsDriver from "@wordup-md/unstorage-drivers/opfs";

const storage = createStorage({ driver: opfsDriver() });
const fs = createGitFs(prefixStorage(storage, "repos:site"));

await git.clone({
  fs,
  http,
  dir: "/",
  url: "https://example.com/site.git",
  corsProxy: "https://cors.isomorphic-git.org",
});
await storage.setItem("repos:site:index.md", "# Edited");
await git.status({ fs, dir: "/", filepath: "index.md" }); // "*modified"

Without a native mtime in the driver meta, the file stats carry a hash of the content, so git only rehashes the files that changed.

Development

vp install   # install dependencies
vp test      # run the tests
vp check     # format, lint and type check
vp pack      # build the library

A new driver goes in src/<name>.ts with an entry in the pack block of vite.config.ts; the package.json exports are generated on build.

License

MIT