flowdb

A lightweight, file-based local database for TypeScript and JavaScript. Data is stored on disk as JSON, with document-style collections, key-value stores, chunked blob storage, backups, and optional encryption.

Features

• File-based persistence: everything lives under a database directory
• Collections: document-style records with generated ids, CRUD, and a fluent query builder
• Key-value stores: simple get / set / remove storage
• Blob storage: store large Buffers in chunks; keep searchable metadata in a collection
• Relations: define hasOne, hasMany, and manyToMany relations and include them in queries
• Backups: snapshot the database directory
• Optional encryption: encrypt collection/store payloads at rest
• Async + sync APIs: most operations have both foo() and fooSync() variants

Requirements

• Node.js (the package is published as ESM; "type": "module" in package.json)

Installation

npm i @kidus.dev/flowdb

Quick start (TypeScript)

import { openDatabase } from "@kidus.dev/flowdb";

const db = openDatabase("my_app", { path: "./data/my_app" });

const users = db.collection("users");
const settings = db.store("settings");

const alice = await users.add({ name: "Alice", age: 30 });
console.log(alice.id);

const found = await users.where("name", { eq: "Alice" }).getFirst();
console.log(found?.raw);

settings.set("theme", "dark");
console.log(settings.get("theme"));

Core concepts

Database

Open a database with openDatabase(name, options).

import { openDatabase } from "@kidus.dev/flowdb";

const db = openDatabase("my_app", {
  path: "./data/my_app", // optional; defaults to the database name
  encrypted: false, // encrypt collection/store file contents
  // secretKey?: string; // (currently only threaded in some drivers)
});

Common methods/properties:

• db.collection(name): get or create a collection
• db.store(name): get or create a key-value store
• db.blobStorage(name, { chunksSize }): get or create a blob storage
• db.backup(name): create a filesystem backup
• db.reset() / db.clear(): delete and recreate the database directory
• db.drop(): delete the database directory entirely
• db.size / db.sizeFormatted: total on-disk size

Collections and documents

A collection holds documents (plain objects) with a stable id.

const posts = db.collection("posts");

// Create
const created = await posts.add({ title: "Hello", published: true });
await posts.addMany([{ title: "First" }, { title: "Second" }]);

// Read
const one = await posts.getById(created.id);
const all = await posts.getAll();
const total = await posts.count();

Export a collection as CSV:

const csv = await posts.exportAsCSV();

Querying

Use collection.where(...) (or collection.builder) to filter, sort, paginate, project fields, and run bulk operations.

const results = await posts
  .where("published", { eq: true })
  .and("views", { gte: 100 })
  .orderBy("title")
  .skip(10)
  .limit(20)
  .get();

const first = await posts.where("title", { startsWith: "Hello" }).getFirst();
const matching = await posts.where("tags", { contains: "js" }).get();

Supported comparison operators (passed as the second arg to where/and/or):

• Equality: eq, neq
• Ordering: gt, gte, lt, lte
• String/list: contains, doesNotContain, startsWith, endsWith, regexMatch
• Ranges: between, betweenStartInclusive, betweenEndInclusive, betweenBothInclusive
• Null/empty: isNull, isNotNull, isEmpty, isNotEmpty
• Custom: filter: (value) => boolean

Other query methods:

• Boolean logic: and(...), or(...)
• Projection: pick("a", "b"), omit("secret")
• Pagination: skip(n), limit(n), take(n), skipAndTake({ skip, take })
• Sorting: orderBy(field, { desc: true })
• Relations: include("relationName")
• Random access: getRandom(), getRandomMany(n), getShuffled()
• Updates: update(data), updateFrom(fn), updateMany(data), updateManyFrom(fn)
• Deletes: delete(), deleteMany()
• Upsert: upsert({ update, insert })
• Insert if absent: addIfAbsent(data)

Sync variants mirror the async API: getSync(), addSync(), countSync(), etc.

Key-value stores

Stores hold JSON-serializable values under string keys.

const cache = db.store("cache");

cache.set("lastSync", new Date().toISOString());
cache.setFrom("counter", (v) => (typeof v === "number" ? v + 1 : 1), 0);

if (cache.has("lastSync")) {
  console.log(cache.get("lastSync"));
}

console.log(cache.all()); // all entries
cache.clear();
cache.drop(); // delete the store from disk

Blob storage

Store large Buffers in chunked files. Blob metadata is stored in a collection (so it’s queryable).

import { Buffer } from "node:buffer";

const files = db.blobStorage("files", { chunksSize: 1024 * 512 });

await files.add(Buffer.from("hello"), { filename: "hello.txt", mime: "text/plain" });

const blobs = await files.get({
  query: (meta) => meta.where("filename", { eq: "hello.txt" }),
});

const firstBlob = blobs[0];
console.log(firstBlob?.metadata.raw);

Relations

Define relations on a collection, then load them via include(...).

const users = db.collection("users");
const posts = db.collection("posts");

// one-to-many: users -> posts (posts.authorId points to users.id)
users.hasMany(posts, "authorId", { localKey: "id", name: "posts" });

// one-to-one
users.hasOne(db.collection("profiles"), "id", { localKey: "profileId", name: "profile" });

// many-to-many
users.manyToMany(db.collection("tags"), "id", { name: "tags" });

// query + include
const withPosts = await users.where("name", { eq: "Alice" }).include("posts").get();

Backups

Backups copy the database directory into backups/<name>/.

const snapshot = db.backup("before-migration");
console.log(db.backups);

db.removeBackup("before-migration");
db.removeAllBackups();

On-disk layout

A database directory typically looks like:

my_app/
├── collections/
│   └── users
├── stores/
│   └── settings
├── blobs/
│   └── files/
│       ├── _metadata/
│       └── chunks/
└── backups/
    └── snapshot_name/

Sync vs async

Use async methods when you don’t want to block I/O; use sync methods in scripts/tests for simplicity.

Examples:

• add / addSync
• getAll / getAllSync
• where(...).get() / where(...).getSync()

Encryption

Pass { encrypted: true } when opening a database to encrypt collection/store file contents at rest:

const db = openDatabase("secure", { path: "./data/secure", encrypted: true });