@diana-db/odm

v2.5.4

Published

22 days ago

ODM for DianaDB

Downloads

829

0High
0Medium
0Low

pashaman

DianaDB

The first fully featured DBMS written in Node.js.

DianaDB is a NoSQL, column-oriented database. It works with documents and collections, but stores data in a highly optimized columnar format: each document is decomposed into separate columns, each column tailored to a specific data type with its own storage and processing logic. Data integrity is enforced through document schemas that define the expected structure of each collection. Because of this layout, every field in a document functions like an index.

If you already know MongoDB, onboarding is quick — the query language is conceptually close to MQL.

| | | | ------------------- | -------------------------------------------------------------- | | Current release | Server v2.5.3 / ODM v2.5.3 | | ODM package | @diana-db/odm | | Documentation | diana-db.com | | License | MIT with a usage restriction — see License | | Maintainer | Data Bikers Limited |

Server >= 2.5.3 requires ODM >= 2.5.4.

Features

| Feature | What it gives you | | ------------------------------ | ------------------------------------------------------------------------------------- | | TLS / mTLS | Encrypt every connection, with optional mutual TLS to verify client identity | | Built for speed | A lean columnar storage engine with no unnecessary overhead between you and your data | | Expressive query language | DiQL describes complex queries clearly and concisely | | Materialized views | Precompute and store query results for instant reads on repeated, demanding queries | | Geo-spatial queries | Query by location, polygon or route natively — no separate geo-indexing service | | Relations | Model connected data naturally without giving up document-store flexibility | | Cross-database lookups | Join data across multiple databases on the same server | | Subscribers | React to database changes in real time, globally or per collection, without polling | | Time queries | Query and manipulate documents by time ranges and intervals as a first-class feature | | ACID transactions | Multi-document transactions with document-level locking | | Built-in migration storage | Track and apply migrations with no extra tooling or external migration table | | S3 sync | Store dumps and backups directly in a private, S3-compatible bucket |

Relations work without JOINs or explicit lookups. Given user and post collections, you can retrieve every post by a user named "John Doe" directly — and every post by users belonging to a specific team, even when the post stores only the user id and the user stores only the team id.

Scheduling work is a single query: every document relating to Tuesdays in July 2025, or shifting all scheduled items to the nearest available slot. That comes from the TIME type and its utilities.

Documents can be updated with commands rather than final scalar values, which makes concurrent updates dynamic and safe.

Quick Start

# 1. Run a server
docker run -p 34567:34567 databikers/diana-db:debian

# 2. Install the client
npm i -s @diana-db/odm

import { DianaDb, Model, Types } from '@diana-db/odm';

const dianaDb = new DianaDb('diana-db://admin:admin@localhost:34567?connectionPoolSize=5');
await dianaDb.connect(5000);

const userModel = new Model({
  database: 'test',
  collection: 'user',
  name: 'User',
  schema: {
    name: { type: Types.STRING, required: true, lowercase: true },
    age: { type: Types.NUMBER, precision: 0 },
    isActive: { type: Types.BOOLEAN, default: true },
    createdAt: { type: Types.TIME, default: () => new Date().toISOString() },
  },
});
await userModel.init();

await userModel.insert({ name: 'John', age: 33 });

const users = await userModel.find([{ name: { $eq: 'john' } }], [], { createdAt: -1 }, 0, 10);

The server creates no default user. Add one yourself and grant access before connecting — see CLI Tool.

Server Installation

Debian

sudo echo "deb [arch=amd64 trusted=yes] https://dist.databikers.com stable main" | sudo tee /etc/apt/sources.list.d/diana-db.list
sudo apt-get update
sudo apt-get install -y diana-db

sudo service diana-db [status|start|stop]

Alpine

wget -q -O /etc/apk/keys/sgerrand.rsa.pub https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub
wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.35-r1/glibc-2.35-r1.apk
apk add glibc-2.35-r1.apk
wget https://dist.databikers.com/x86_64/diana-db-2.5.3-r0.apk
apk add --allow-untrusted ./diana-db-2.5.3-r0.apk

sudo rc-service diana-db [status|start|stop]

Docker

entrypoint.sh — Debian image:

#!/bin/sh
set -e
touch /var/lib/diana-db/current.didb
touch /var/lib/diana-db/users
/usr/bin/diana-db add-user -u admin -p admin
/usr/bin/diana-db grant-access -u admin -db test:3
/usr/bin/diana-db-server

entrypoint.sh — Alpine image:

#!/bin/sh
set -e
/usr/bin/diana-db add-user -u admin -p admin
/usr/bin/diana-db grant-access -u admin -db test:3
exec /usr/bin/diana-db-server

Dockerfile:

FROM databikers/diana-db:debian
COPY entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]

docker build -t local-diana-db .
docker run -p 34567:34567 local-diana-db

docker-compose.yml:

version: '3.8'
services:
  diana-db:
    image: local-diana-db
    ports:
      - '34567:34567'
    volumes:
      - ./data/etc:/etc/diana-db
      - ./data/lib:/var/lib/diana-db
      - ./data/log:/var/log/diana-db
    restart: unless-stopped
    container_name: diana-db
    entrypoint: ['/entrypoint.sh']

CLI Tool

diana-db manages the daemon, users, access and backups.

| Command | Flags | Description | | --------------- | --------------------------- | ----------------------------------------------------------------------------- | | settings | – | Displays current settings (diana-db --settings) | | version | – | Displays current version (diana-db --version) | | start | -c <conf> | Stops any running instance, then launches diana-db-server in the background | | stop | -c <conf> | Kills the running diana-db-server process | | add-user | -u <user> -p <pass> | Creates a user with the given credentials | | remove-user | -u <user> | Removes the user | | grant-access | -u <user> -db <db:access> | Grants database access. Levels: 0 read, 1 write, 3 manage | | remove-access | -u <user> -db <db> | Removes database access | | list-backups | – | Displays the backup list | | create-backup | -n <name> | Creates [name].didb. Server must be stopped | | apply-backup | -n <name> | Applies [name].didb. Server must be stopped | | remove-backup | -n <name> | Removes [name].didb |

diana-db start -c /path/to/diana-db.conf
diana-db add-user -u alice -p s3cr3t
diana-db grant-access -u alice -db main:3
diana-db create-backup -n test_backup

Configuration

diana-db reads a key=value config, defaulting to /etc/diana-db/diana-db.conf. Each key maps to an environment variable used by diana-db-server. Comments start with #.

| Key | -------------------------------------- | port | dump_create_interval | logs_directory | dump_directory | logs_ttl_value | backup_name | current_dump_name | store_strategy | store_s3_endpoint | store_s3_bucket | store_s3_region | store_s3_access_key_id | store_s3_secret_access_key | ssl_cert_file | ssl_key_file | ssl_ca_file | ssl_request_cert | ssl_reject_unauthorized | transactions_min_auto_ | transactions_max_auto_ | Environment Variable | Type | Description | | --------------------------------------- | ------------- | ------------------------------------------------ | | DIANA_DB_PORT | Integer | Listening TCP port (default 34567) | | DIANA_DB_DUMP_CREATE_INTERVAL | Integer (ms) | How often to snapshot | | DIANA_DB_LOG_DIRECTORY | Path | Server log directory | | DIANA_DB_DUMP_DIRECTORY | Path | Dump directory | | DIANA_DB_LOG_TTL | Integer (ms) | Log retention | | DIANA_DB_BACKUP_NAME | String | Default backup filename when -n is omitted | | DIANA_DB_DUMP_CURRENT_NAME | String | Filename for the latest dump | | DIANA_DB_STORE_STRATEGY | String | local (default) or s3 | | DIANA_DB_STORE_S3_ENDPOINT | URL | Required when store_strategy=s3 | | DIANA_DB_STORE_S3_BUCKET | String | Required when store_strategy=s3 | | DIANA_DB_STORE_S3_REGION | String | Required when store_strategy=s3 | | DIANA_DB_STORE_S3_ACCESS_KEY_ID | String | Required when store_strategy=s3 | | DIANA_DB_STORE_S3_SECRET_ACCESS_KEY | String | Required when store_strategy=s3 | | DIANA_DB_SSL_CERT_FILE | Path | TLS certificate. Enables TLS with ssl_key_file | | DIANA_DB_SSL_KEY_FILE | Path | TLS private key | | DIANA_DB_SSL_CA_FILE | Path | CA used to verify client certificates (mTLS) | | DIANA_DB_SSL_REQUEST_CERT | Boolean (0/1) | Request a client certificate (mTLS) | | DIANA_DB_SSL_REJECT_UNAUTHORIZED | Boolean (0/1) | Reject certificates that fail verification | rollback_value | DIANA_DB_TRANSACTION_AUTOROLLBACK_MIN | Integer (ms) | Minimum auto-rollback delay | rollback_value | DIANA_DB_TRANSACTION_AUTOROLLBACK_MAX | Integer (ms) | Maximum auto-rollback delay |

# /etc/diana-db/diana-db.conf
binary_path=/usr/bin/diana-db-server
port=34567
transactions_min_auto_rollback_value=1000
transactions_max_auto_rollback_value=60000
logs_directory=/var/log/diana-db
logs_ttl_value=7d
dump_directory=/var/lib/diana-db
current_dump_name=current.didb
backup_name=backup.didb
dump_create_interval=day
store_strategy=local

Switching to store_strategy=s3: upload your existing users file and current data (/var/lib/diana-db/*) to the target bucket first. DianaDB does not migrate local data automatically — switching without uploading starts the server with an empty dataset.

Security

TLS / mTLS. Connections can be encrypted in transit, with optional mutual TLS so the server verifies client identity via certificates.
The password is never transmitted. The client identifies itself with a username; every request and response is then encrypted with AES-256-GCM using a key derived from the user's password. Tampered or wrongly-keyed requests fail authentication and are rejected.
Segregated access control. Access is granted per user and per database — read, write, or manage.
Private storage. Dumps and backups can be written to a private S3-compatible bucket.

Passwords are never sent over the wire, but connection metadata is sent in plaintext at connection start. Enable TLS if metadata confidentiality matters.

Desktop Application

A cross-platform GUI for working directly against a server: browse and manage data, edit schemas and their relationships, create and inspect materialized views, and test queries against real data.

Linux (AppImage): https://dist.databikers.com/gui/diana-db-gui/DianaDB-GUI-1.1.2.AppImage
Windows: https://dist.databikers.com/gui/diana-db-gui/DianaDB-GUI%20Setup%201.1.2.exe

ODM for Node.js

npm i -s @diana-db/odm

Connecting

import { readFileSync } from 'node:fs';
import { DianaDb } from '@diana-db/odm';

export const dianaDb = new DianaDb({
  host: 'localhost',
  port: 34567,
  user: 'db-user',
  password: 'some-password',
  connectionPoolSize: 5,
  connectTimeoutValue: 5000,
  reconnectTimeoutValue: 1000,
  secureServer: true,
  tls: {
    cert: readFileSync('/path/to/cert'),
    key: readFileSync('/path/to/key'),
    ca: readFileSync('/path/to/ca'),
    rejectUnauthorized: true,
  },
});

await dianaDb.connect(5000);

Or with a connection string:

new DianaDb('diana-db://user:pass@localhost:34567?connectionPoolSize=5'); // plain
new DianaDb('diana-dbs://user:pass@localhost:34567?connectionPoolSize=5'); // TLS

connectionPoolSize, connectTimeoutValue and reconnectTimeoutValue are accepted as query parameters. The diana-dbs:// scheme sets secureServer: true; any other protocol throws a configuration error.

You can create as many instances as you like — each has its own models, migrations and subscribers, and connects independently.

Constructor Options

| Option | Type | Default | Constraint | | ----------------------- | --------------------------------------------------- | --------- | ------------------------ | | host | string | – | required | | port | integer | 34567 | required, 1025–65535 | | user | string | – | required | | password | string | – | required | | connectionPoolSize | integer | 5 | >= 1 | | connectTimeoutValue | integer (ms) | 30000 | >= 100 | | reconnectTimeoutValue | integer (ms) | 1000 | >= 100 | | secureServer | boolean | false | – | | tls | { cert, key, ca?, rejectUnauthorized? } (Buffers) | – | – | | logger | object | console | – |

Client Methods

| Method | Arguments | Returns | | ------------------------ | -------------------------------------------------------- | ----------------------------------------------- | | connect | connectTimeoutValue: number | Promise<boolean> | | disconnect | – | Promise<void> | | subscribe | key: string, subscriber: (u: DatabaseUpdate) => void | Promise<void> | | addMigration | migration: Migration | void | | migrateUp | – | Promise<void> | | migrateDown | – | Promise<void> | | startTransaction | { database, autoRollBackAfterMs? } | Promise<string> | | commitTransaction | { database, transactionId } | Promise<TransactionInfo> | | rollbackTransaction | { database, transactionId } | Promise<TransactionInfo> | | getDatabases | – | Promise<string[]> | | createDatabase | database | Promise<{ database, action }> | | removeDatabase | database | Promise<{ database, action }> | | getCollections | database | Promise<string[]> | | createCollection | database, collection, schema | Promise<{ collection, action }> | | removeCollection | database, collection | Promise<{ collection, action }> | | getCollectionSchema | database, collection | Promise<{ schema, locked }> | | lockCollectionSchema | database, collection | Promise<{ collection, locked }> | | unlockCollectionSchema | database, collection | Promise<{ collection, locked }> | | getCollectionViews | database, collection | Promise<Record<string, { transformQueries }>> | | getHealth | – | Promise<HealthInfo> | | getVersion | – | Promise<{ version: string }> |

type HealthInfo = {
  memory: number; // MB
  cpu: number; // %
  gcp: number; // garbage collector pressure, %
  elu: number; // event loop utilization, %
  eld: number; // event loop delay, ms
};

Document Schema

A schema is a configuration object describing the structure of stored documents. Keys map to document properties. _id is added automatically — you do not declare it.

Field Options

| Option | Type | Description | Required | Default | | --------------- | ---------------------------------- | ----------------------------------------------------------------------------- | --------------------------------------- | ------- | | type | Types | Field type | Yes | – | | required | boolean | Property must be present | No | false | | unique | boolean | Property must be unique | No | false | | mutable | boolean | Value may change after creation | No | true | | lowercase | boolean | STRING only. Lowercases the value | No | false | | uppercase | boolean | STRING only. Uppercases the value | No | false | | default | T, () => T, () => Promise<T> | Default value or factory, evaluated in the document's context | No | – | | set | function | Applied to the incoming value on insert and update, before validation | No | – | | get | function | Applied to each found document. Skipped when the query uses transform queries | No | – | | items | Types | ARRAY only. Item type | Yes (if ARRAY) | – | | reference | string | Referenced collection name | Yes (if type or items is REFERENCE) | – | | triggerRemove | boolean | REFERENCE only. Remove this document when the referenced item is deleted | No | false | | ttl | integer >= 5 | TIME only. Time to live in ms from insertion | No | – | | precision | integer 0–8 | NUMBER only. Decimal places retained | Yes (if NUMBER) | – |

Enforced exclusions:

lowercase and uppercase cannot both be set.
At most one of default, set, get per field.
Type-scoped options are rejected on other types — this is a validation error, not a silently ignored option.

Field Types

| Type | Description | Notes | | ----------------- | -------------------------------------------- | ------------------------------------------------------------------ | | Types.STRING | String | Supports lowercase / uppercase | | Types.NUMBER | Number | Requires precision | | Types.BOOLEAN | Boolean | – | | Types.POSITION | { x, y } point | Enables spatial queries | | Types.OBJECT_ID | Hex string identifier | Not BSON. /^[0-9a-f]{12}-[0-9a-f]{1,}-[0-9a-f]{11}-[0-9a-f]{9}$/ | | Types.REFERENCE | Like OBJECT_ID, linked to another collection | Requires reference. Enables $subQuery | | Types.TIME | ISO 8601 string or epoch milliseconds | Supports time filters and ttl | | Types.ARRAY | Homogeneous array of items type | Any type as items. Set-like: unique values only |

import { Types, Schema } from '@diana-db/odm';

export const userSchema: Schema<User> = {
  name: { type: Types.STRING, unique: true, required: true, lowercase: true },
  isActive: { type: Types.BOOLEAN, default: true },
  position: { type: Types.POSITION },
  createdAt: { type: Types.TIME, default: () => new Date().toISOString() },
};

export const postSchema: Schema<Post> = {
  user: { type: Types.REFERENCE, reference: 'user', required: true, triggerRemove: true },
  title: { type: Types.STRING, required: true },
  isPublished: { type: Types.BOOLEAN, default: true },
  publishedAt: { type: Types.TIME, required: true },
};

Be careful with schema changes. Differences between the old and new schema are detected automatically and applied server-side when your code runs — you may lose data. Keep schemas in one shared package used as a dependency across your project, or lock the collection schema.

Model

import { Model } from '@diana-db/odm';

const userModel = new Model<User>({
  database: 'test',
  collection: 'user',
  name: 'User',
  schema: userSchema,
});

await userModel.init();

| Method | Description | Returns | | ------------------------------------------------------------------ | ------------------------------------------------------------------------------- | ----------------------------------------------- | | init() | Creates the collection or reconciles the schema. Called lazily by other methods | Promise<void> | | Model.init() (static) | Initializes every constructed model | Promise<void> | | insert(data, transactionId?) | Applies defaults and setters, validates, inserts | Promise<T & { _id }> | | find(filters, transforms?, sort?, skip?, limit?, transactionId?) | Finds documents | Promise<T[]> | | count(filters, transforms?, transactionId?) | Counts matching documents | Promise<number> | | update(filters, updateData, transactionId?) | Updates matching documents | Promise<{ found, modified }> | | remove(filters, transactionId?) | Removes matching documents | Promise<{ found, removed }> | | distinct(key) | Unique values for a key | Promise<any[]> | | max(key) | Documents holding the maximum value | Promise<T[]> | | min(key) | Documents holding the minimum value | Promise<T[]> | | createView(name, transformQueries) | Creates a materialized view | Promise<{ view, action: 'created' }> | | removeView(name) | Removes a materialized view | Promise<{ view, action: 'removed' }> | | getViews() | Views defined on this collection | Promise<Record<string, { transformQueries }>> | | findByView(name, filter?, sort?, skip?, limit?) | Queries a view | Promise<any[]> | | countByView(name, filter?) | Counts results in a view | Promise<number> | | lock() | Locks this collection's schema | Promise<{ collection, locked }> | | unlock() | Unlocks this collection's schema | Promise<{ collection, locked }> |

Insert

const user = await userModel.insert({
  name: 'John',
  position: { x: 1, y: 1 },
  createdAt: new Date().toISOString(),
});

{
  _id: 'dc4628514b81-939f-198a25f76b7-bb1a9a777',
  name: 'john',
  position: { x: 1, y: 1 },
  isActive: true,
  createdAt: '2025-06-18T06:30:24.977Z'
}

_id is added automatically. Fields with default are filled when absent; fields with set pass through the setter before validation. The returned document is a plain object with no special methods.

Find and Count

const [user] = await userModel.find(
  [{ name: { $eq: 'John' } }], // FindQuery — array behaves as OR
  [{ $project: { name: true } }], // Transform Queries
  { _id: -1 }, // Sorting
  0, // Skip
  1, // Limit
);

Operands within one FindQuery are ANDed; an array of FindQueries is ORed. The first argument is required, the rest are optional. count takes the first two arguments plus transactionId and returns a number.

Query Operands

| Operand | Types | Description | | --------------------------------------------------------- | -------------------- | ---------------------------------------------- | | $eq | all | Equals. Shorthand: { name: 'John' } | | $ne | all | Not equal | | $in / $nin | all | Value is / is not in the given array | | $gt $gte $lt $lte | NUMBER | Range comparison | | $regex | STRING | Regular expression match | | $startsWith $endsWith $notStartsWith $notEndsWith | STRING | Prefix / suffix match | | $cn / $nc | STRING | Contains / does not contain | | $subQuery | REFERENCE, OBJECT_ID | Nested query against the referenced collection |

On OBJECT_ID, REFERENCE and BOOLEAN fields exactly one operand may be used per field, so $subQuery must stand alone.

TIME Queries

A TIME field is never compared directly. Each part operand takes a numeric query; $raw takes a string query against the stored value.

$year $month $date $hours $minutes $seconds $dayOfWeek $dayOfYear $timestamp $raw

// Range over an instant
{ createdAt: { $timestamp: { $gte: 1750000000000, $lt: 1755000000000 } } }

// Calendar parts
{ createdAt: { $year: { $eq: 2025 }, $month: { $in: [1, 4] }, $dayOfWeek: { $in: [0, 6] } } }

// Prefix match on the stored ISO value
{ createdAt: { $raw: { $startsWith: '2025-06' } } }

Spatial Queries

{ position: { $insideCircle: { center: { x: 1, y: 1 }, radius: 10 } } }
{ position: { $insidePolygon: [{ x: 1, y: 1 }, { x: 3, y: 0 }, { x: -1, y: -1 }] } }
{ position: { $nearLines: { lines: [[{ x: 1, y: 1 }, { x: 10, y: 10 }]], distance: 10 } } }

$insideCircle $outsideCircle $insidePolygon $outsidePolygon $nearLines $farFromLines. Polygons need at least 3 points; each line is exactly 2 points, at least one line. $nearLines.distance must be positive; $farFromLines.distance may be 0.

Array and Reference Queries

Array fields accept the same operands as their item type, applied per element with OR semantics:

// documents whose array contains 1 and 2 but not 3
{ someArrayProperty: { $in: [1, 2], $nin: [3] } }

Reference fields query the target collection inline, with no aggregation:

{
  user: {
    $subQuery: {
      name: {
        $eq: 'John';
      }
    }
  }
}

distinct / max / min

const names = await userModel.distinct('name');
const best = await userModel.max('score');
const worst = await userModel.min('score');

Transform Queries

An array of stage objects, each holding exactly one key.

| Stage | Description | | -------------- | ---------------------------------------------------------------------------------------------------------------------- | | $project | Each key becomes a property of the result. Value: boolean (keep/drop), pointer string, or a projection operand object | | $group | Same shape as $project plus a required _id — false to merge into one, a pointer string, or an object of pointers | | $match | Filters current documents. Boolean, string, number, TIME and spatial queries only — not $subQuery or _id | | $lookup | { collection*, as*, localField*, foreignField*, database, filter, sort, skip, limit, replaceRoot } | | $unwind | Pointer string. One document per array item | | $replaceRoot | Pointer string. Replaces each document with the value at that property | | $sort | { field: 1 \| -1 }, multi-key | | $skip | Integer >= 0 | | $limit | Positive integer, >= 1 |

$unwind and $replaceRoot values are pointers — they start with $.

const result = await postModel.find(
  [{}],
  [
    { $group: { _id: '$user', user: { $first: '$user' }, postsCount: { $sum: [1] } } },
    {
      $lookup: {
        database: 'test',
        collection: 'user',
        localField: 'user',
        foreignField: '_id',
        as: 'user',
        filter: { isActive: { $eq: true } },
      },
    },
    { $unwind: '$user' },
    { $project: { _id: false, user: true, postsCount: true } },
    { $sort: { postsCount: -1 } },
    { $skip: 1 },
    { $limit: 1 },
  ],
);

Projection Operands

Each projection object holds exactly one operand. Since Server 1.4.8 math operands never implicitly include the existing value — reference it as a pointer: { $sum: ['$amount', 123] }.

| Operand | Format | Description | | ---------------------------------------------------------------------------------------------- | ----------------------------- | ------------------------------------------------------- | | $max $min $avg | pointer string | Maximum / minimum / average | | $sum $subtract $multiply $divide | array of numbers and pointers | Arithmetic. $divide rejects a 0 divisor | | $round | integer 0–8 | Rounds to the given precision | | $ifNull | array | First non-null / non-undefined value | | $push | array | Creates an array property and pushes values | | $addToSet | array | Like $push, unique only | | $concatArray | pointer to an array property | Concatenates arrays | | $first $last | pointer string | First / last value | | $concat | { delimiter, parts[] } | Concatenates strings; pointers in parts are evaluated | | $year $month $date $hours $minutes $seconds $dayOfWeek $dayOfYear $timestamp | pointer to a TIME property | Extracts the part as a number |

Update

await userModel.update([{ name: { $eq: 'John' } }], { isActive: false });

Values may be plain values, or an operator object describing an update strategy for that field's type.

String

| Operator | Type | Effect | | ---------- | ------------------------------ | ------------------------------------------------------------------------- | | $concat | { delimiter, parts[] } | Concatenates parts with delimiter | | $replace | [search, replacement, flags] | Replaces a substring; flags is a RegExp flag combination such as 'gm' |

Number — each operand accepts a literal or a pointer string

| Operator | Effect | | ------------------------------ | ---------------------------------- | | $add $subtract $multiply | Arithmetic on the current value | | $divide | Divides. Divisor may not be 0 | | $round | Rounds to a precision, integer 0–8 |

Array — exactly one operator per field per call

| Operator | Effect | | ------------- | ----------------------------------------- | | $addItem | Appends an element if not already present | | $removeItem | Removes an element if found |

Renamed in 1.9.0 from $add / $remove to avoid clashing with the numeric operators.

TIME

| Operator | Structure | Effect | | ----------------------------------- | ------------------------------------------------ | ------------------------------------------------------- | | $add / $subtract | { amount: positive integer, unit: UnitOfTime } | Adds / subtracts time | | $toStartOf / $toEndOf | UnitOfTime | Rounds down / up to the unit | | $toNextWeekday / $toLastWeekday | Weekday | Jumps to the next / previous occurrence of that weekday |

UnitOfTime: 'year' 'month' 'week' 'day' 'hour' 'minute' 'second' 'millisecond'

Weekday: 'Sunday' 'Monday' 'Tuesday' 'Wednesday' 'Thursday' 'Friday' 'Saturday'

Remove

const { found, removed } = await userModel.remove([{ isActive: { $eq: false } }]);

Materialized Views

A view is a stored aggregation that updates automatically when the underlying data changes. Views support cross-database lookups.

await transactionModel.createView('balance', [
  {
    $group: {
      _id: { user: '$user', currency: '$currency' },
      user: { $first: '$user' },
      amount: { $sum: ['$amount'] },
      currency: { $first: '$currency' },
    },
  },
]);

const balances = await transactionModel.findByView('balance', { user: { $eq: userId } }, { amount: -1 });
const count = await transactionModel.countByView('balance', { user: { $eq: userId } });
const views = await transactionModel.getViews();

await transactionModel.removeView('balance');

The view name must not collide with a collection or view name in that database. Keys in a view's findQuery match the view's output shape, not the source schema.

Subscribers

// 'user.post' = database 'user', collection 'post'
// 'user' alone = every collection in the 'user' database
await dianaDb.subscribe('user.post', (databaseUpdate) => {
  // handle the update
});

| Field | Type | Description | | ------------- | ---------------------------------- | --------------------------------------- | | database | string | Database name | | collection | string | Collection name | | action | 'insert' \| 'update' \| 'remove' | Operation that fired the update | | affectedIds | string[] | Ids affected | | data | object | Document snapshot for insert and update |

Transactions

Non-blocking ACID transactions with document-level (row-level) locking.

Transactions have higher priority than regular requests and can overwrite previously saved data.
Affected documents are locked dynamically throughout execution until commit or rollback.
Each transaction has its own scope; changes are invisible outside it until committed.

const transactionId = await dianaDb.startTransaction({
  database: 'test',
  autoRollBackAfterMs: 60000,
});

const manageTransactionParameters = { database: 'test', transactionId };

try {
  await userModel.remove([{ ...filters }], transactionId);
  await dianaDb.commitTransaction(manageTransactionParameters);
} catch (e) {
  await dianaDb.rollbackTransaction(manageTransactionParameters);
}

autoRollBackAfterMs is clamped by the server's transactions_min_auto_rollback_value and transactions_max_auto_rollback_value. Both calls resolve to { transactionId, status }, where status is 'committed' or 'rolled-back'.

Migrations

DianaDB ships its own migration storage.

dianaDb.addMigration({
  index: 1, // unique integer index
  name: 'MyFirstMigration', // unique name
  up: async () => {
    // apply changes
  },
  down: async () => {
    // revert changes
  },
});

await dianaDb.connect(5000);
await dianaDb.migrateUp();
await dianaDb.migrateDown();

A migration may span many databases and collections. There is a single list of applied migrations, so names and indexes must be unique — a duplicate index throws immediately. Migrations run in ascending index order. Never edit the code inside a migration after it has run.

Cook Book

An end-to-end mini-shop example spanning seven collections — user, userLocation, item, price, storeItem, order, transaction — and seven materialized views:

currentPrice — latest price per item
inventory — stock count aware of reserved units
balanceActual, balancePendingDeposits, balancePendingWithdrawals — three-way balance per user and currency
orderToPay, orderToShip — orders joined to user and location via $lookup

The flow seeds users, locations, items, prices, stock and deposits, then places an order that reserves store items inside an ACID transaction: if the balance covers the total the order ships immediately, otherwise it stays in toPay. orderToShip can then be queried by location with spatial operands.

Full walkthrough: https://diana-db.com/cookbook/shop-example

Benchmarks

10,000 documents, inserted one at a time and awaited sequentially. Query filters by name, sorts by _id descending, skips 100 and limits 10. All three servers in Docker on one host.

| Database | Insert (10,000) | Insert throughput | Query | Query vs. DianaDB | | ----------- | --------------- | ----------------- | ------------ | ----------------- | | DianaDB | 5.290 s | 1,890 docs/s | 1.959 ms | baseline | | MongoDB | 1.518 s | 6,588 docs/s | 16.683 ms | 8.5× slower | | PostgreSQL | 7.122 s | 1,404 rows/s | 7.114 ms | 3.6× slower |

MongoDB leads on write throughput; DianaDB sits between MongoDB and PostgreSQL. On the read path the columnar layout shows — the filter on name needs no separate index, because the field is the index.

Single run, no warm-up, no repeated runs, no tuning of any engine. Treat it as an order-of-magnitude comparison of an out-of-the-box setup, and run it yourself: databikers/db-benchmark

Versioning and Changelog

Server and ODM versions are released in step and must be kept compatible.

| Boundary | Requirement | | --------------- | ------------ | | Server >= 2.0.0 | ODM >= 2.0.0 | | Server >= 1.7.0 | ODM >= 1.7.0 |

Recent highlights:

v2.5.3 — client-level database management (createDatabase, removeDatabase, createCollection, removeCollection), replaceRoot in LookupQuery, core optimizations
v2.2.1 — getVersion()
v2.1.3 / ODM 2.1.0 — removeView, getViews, getCollectionViews
v2.1.2 / ODM 2.0.0 — TLS/mTLS, local and s3 storage strategies, schema locking, document-level transaction locking (breaking)
v1.9.2 — getDatabases(), getHealth()
v1.9.1 — uppercase / lowercase string options
v1.9.0 — array operators renamed to $addItem / $removeItem
v1.8.3 — countByView
v1.8.1 — connection string support
v1.6.0 — materialized views updating on the fly, access and backup management in the CLI
v1.5.0 — max / min methods and $max / $min / $avg operators
v1.4.8 — distinct method

Full history: https://diana-db.com/changes-log

Roadmap

Desktop Management Tool — GUI for managing your databases
Query Language Improvements — making DiQL more expressive and powerful
Golang, PHP, C#, Java clients — official clients to extend integration across ecosystems
Vectors — vector data type and similarity search
Multi-Server Replication — real-time synchronization and high availability across nodes in different data centres

Reporting Issues

Open an issue at https://github.com/databikers/diana-db/issues. Include your server and ODM versions, the schema involved, and a minimal reproduction where possible.

License

Permission is granted, free of charge, to use, copy, modify, merge, publish, distribute, sublicense and sell copies of the Software, subject to:

The copyright notice and this permission notice being included in all copies or substantial portions of the Software.
The Software may not be used to create or offer a competing product or service without prior written consent from Data Bikers Limited.

The Software is provided "as is", without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose and non-infringement.