MiniQDB

A lightweight, JSON-based database engine for Node.js with ACID transactions, schema validation, indexing, and HTTP server capabilities.

Features

• Collection-Based Storage: Organize data into named collections with optional schema validation
• Schema Validation: Define strict schemas with required fields, types, defaults, and custom validators
• ACID Transactions: Full transaction support with commit and rollback capabilities
• Indexing: Create indexes on fields for optimized query performance
• Advanced Querying: Support for complex filters, comparison operators, regex, and logical operators
• Aggregation Pipeline: MongoDB-like aggregation with stages like $match, $group, $lookup, $project, and more
• Update Operators: Use operators like $set, $inc, $push, $pull, $min, $max, etc.
• HTTP Server: Built-in REST API server with CORS support
• Atomic Writes: Ensures data integrity with atomic file operations
• Bulk Operations: Batch insert, update, and delete operations

Installation

npm install miniqdb

Quick Start

const { MiniQDB } = require("miniqdb");

// Create a new database instance
const Path = "db.json";

const db = new MiniQDB(Path);

// Get or create a collection
const users = db.collection("users", {
  name: { type: "string", required: true },
  email: { type: "string", unique: true },
  age: { type: "number", min: 0, max: 150 },
  createdAt: { type: "string", default: () => new Date().toISOString() },
});

// Insert a document
const result = users.insertOne({
  name: "John Doe",
  email: "[email protected]",
  age: 30,
});

console.log("Inserted:", result.insertedItem);

// Find documents
const docs = users.find({ age: { $gte: 18 } });
console.log("Adults:", docs);

// Update a document
users.updateOne({ email: "[email protected]" }, { $set: { age: 31 } });

// Delete a document
users.deleteOne({ email: "[email protected]" });

Database Path

By default, MiniQDB stores data in db/data.json. You can specify a custom path:

const db = new MiniQDB("/path/to/custom/database.json");

Collections

Creating a Collection

const collection = db.collection(name, schema, options);

Parameters:

• name (string): Collection name
• schema (object): Optional schema definition
• options (object): Optional configuration
  • strict (boolean): Enforce schema strictly (default: false)

Schema Definition

const schema = {
  username: {
    type: "string",
    required: true,
    unique: true,
    minLength: 3,
    maxLength: 20,
  },
  email: {
    type: "string",
    unique: true,
    pattern: "^[^@]+@[^@]+\\.[^@]+$",
  },
  age: {
    type: "number",
    min: 0,
    max: 150,
  },
  status: {
    type: "string",
    enum: ["active", "inactive", "suspended"],
    default: "active",
  },
  tags: {
    type: "array",
    items: { type: "string" },
  },
  profile: {
    type: "object",
    properties: {
      firstName: { type: "string", required: true },
      lastName: { type: "string", required: true },
    },
  },
};

CRUD Operations

Insert

// Insert one document
const result = collection.insertOne({
  name: "Alice",
  email: "[email protected]",
});

// Insert multiple documents
const results = collection.insertMany([
  { name: "Bob", email: "[email protected]" },
  { name: "Charlie", email: "[email protected]" },
]);

Find

// Find all documents
const all = collection.find();

// Find with filter
const adults = collection.find({ age: { $gte: 18 } });

// Find with options
const paginated = collection.find(
  { status: "active" },
  {
    sort: { createdAt: -1 },
    limit: 10,
    skip: 20,
    projection: { password: 0 }, // Exclude password field
  },
);

// Find one document
const user = collection.findOne({ email: "[email protected]" });

// Find by ID
const doc = collection.findById("document_id");

Update

// Update one document
collection.updateOne({ email: "[email protected]" }, { $set: { age: 25 } });

// Update many documents
collection.updateMany(
  { status: "inactive" },
  { $set: { status: "suspended" } },
);

// Replace one document
collection.replaceOne(
  { _id: "doc_id" },
  { name: "Alice Updated", email: "[email protected]" },
  { upsert: true },
);

// Find and update (returns updated document)
const updated = collection.findOneAndUpdate(
  { email: "[email protected]" },
  { $inc: { loginCount: 1 } },
  { returnOriginal: false },
);

Delete

// Delete one document
const result = collection.deleteOne({ email: "[email protected]" });

// Delete many documents
const results = collection.deleteMany({ status: "inactive" });

// Find and delete (returns deleted document)
const deleted = collection.findOneAndDelete({ _id: "doc_id" });

Query Operators

Comparison Operators

// Equal
collection.find({ age: { $eq: 30 } });

// Not equal
collection.find({ status: { $ne: "inactive" } });

// Greater than
collection.find({ age: { $gt: 18 } });

// Greater than or equal
collection.find({ age: { $gte: 18 } });

// Less than
collection.find({ age: { $lt: 65 } });

// Less than or equal
collection.find({ age: { $lte: 65 } });

// In array
collection.find({ status: { $in: ["active", "pending"] } });

// Not in array
collection.find({ status: { $nin: ["deleted", "banned"] } });

Logical Operators

// OR
collection.find({ $or: [{ age: { $lt: 18 } }, { age: { $gt: 65 } }] });

// AND
collection.find({ $and: [{ status: "active" }, { verified: true }] });

// NOR
collection.find({ $nor: [{ banned: true }, { suspended: true }] });

// NOT
collection.find({ $not: { age: { $lt: 18 } } });

String Operators

// Regex match
collection.find({ email: { $regex: "^admin.*@example\\.com$" } });

// Regex with options (case-insensitive)
collection.find({ name: { $regex: "john", $options: "i" } });

Array Operators

// Element match
collection.find({ tags: { $elemMatch: { $eq: "javascript" } } });

// All elements
collection.find({ tags: { $all: ["javascript", "nodejs"] } });

// Size
collection.find({ tags: { $size: 3 } });

Other Operators

// Field exists
collection.find({ email: { $exists: true } });

// Type check
collection.find({ age: { $type: "number" } });

Update Operators

// Set field value
collection.updateOne({ _id: "id" }, { $set: { status: "active" } });

// Increment numeric field
collection.updateOne({ _id: "id" }, { $inc: { views: 1 } });

// Multiply numeric field
collection.updateOne({ _id: "id" }, { $mul: { price: 1.1 } });

// Minimum value
collection.updateOne({ _id: "id" }, { $min: { score: 50 } });

// Maximum value
collection.updateOne({ _id: "id" }, { $max: { score: 100 } });

// Unset field
collection.updateOne({ _id: "id" }, { $unset: { tempField: true } });

// Rename field
collection.updateOne({ _id: "id" }, { $rename: { oldName: "newName" } });

// Set current date
collection.updateOne({ _id: "id" }, { $currentDate: { lastUpdated: true } });

// Push to array
collection.updateOne({ _id: "id" }, { $push: { tags: "newtag" } });

// Push multiple values
collection.updateOne(
  { _id: "id" },
  { $push: { tags: { $each: ["tag1", "tag2"] } } },
);

// Pull from array
collection.updateOne({ _id: "id" }, { $pull: { tags: "oldtag" } });

Aggregation Pipeline

const pipeline = [
  { $match: { status: "active" } },
  {
    $group: {
      _id: "$department",
      count: { $sum: 1 },
      avgSalary: { $avg: "$salary" },
    },
  },
  { $sort: { count: -1 } },
  { $limit: 10 },
];

const results = collection.aggregate(pipeline);

Aggregation Stages

• $match: Filter documents
• $project: Include/exclude fields
• $group: Group and accumulate
• $sort: Sort results
• $limit: Limit result count
• $skip: Skip documents
• $unwind: Unwrap array fields
• $lookup: Join with other collections
• $addFields/$set: Add new fields
• $unset: Remove fields
• $replaceRoot: Replace root document
• $count: Count documents
• $facet: Multi-stage processing
• $bucket: Bucket/bin documents

Indexing

// Create an index on a field
collection.createIndex("email");

// This speeds up queries like:
collection.find({ email: "[email protected]" });

// Drop an index
collection.dropIndex("email");

Transactions

try {
  db.beginTransaction();

  collection1.insertOne({ data: "value1" });
  collection2.updateOne({ _id: "id" }, { $set: { data: "value2" } });

  db.commit();
} catch (error) {
  db.rollback();
  console.error("Transaction failed:", error);
}

Bulk Operations

const operations = [
  { insertOne: { document: { name: "User1" } } },
  {
    updateOne: {
      filter: { _id: "id1" },
      update: { $set: { status: "active" } },
    },
  },
  { deleteOne: { filter: { _id: "id2" } } },
];

const result = collection.bulkWrite(operations);
console.log(
  `Inserted: ${result.insertedCount}, Modified: ${result.modifiedCount}, Deleted: ${result.deletedCount}`,
);

HTTP Server

const { MiniQDB } = require("miniqdb");

const Path = "db.json";
const db = new MiniQDB(Path);

// Start HTTP server on port 3000
db.listen(3000, {
  maxBodySize: 1024 * 1024, // Max request body size (1MB)
  allowedCollections: ["users", "posts"], // Whitelist collections
});

API Endpoints

GET /                           - List all collections
GET /collection                 - Get all documents
POST /collection                - Insert document or aggregate
GET /collection/:id             - Get document by ID
PUT /collection/:id             - Replace document (upsert)
PATCH /collection/:id           - Update document (upsert)
DELETE /collection/:id          - Delete document

Example Requests

# List collections
curl http://localhost:3000

# Get all users
curl http://localhost:3000/users

# Query with filters
curl "http://localhost:3000/users?age=\$gte:30&status=active"

# Pagination
curl "http://localhost:3000/users?_limit=10&_skip=20"

# Sorting
curl "http://localhost:3000/users?_sort=createdAt&_order=desc"

# Insert document
curl -X POST http://localhost:3000/users \
  -H "Content-Type: application/json" \
  -d '{"name":"John","email":"[email protected]"}'

# Aggregation
curl -X POST http://localhost:3000/users \
  -H "Content-Type: application/json" \
  -d '{
    "pipeline": [
      {"$match": {"status": "active"}},
      {"$group": {"_id": "$department", "count": {"$sum": 1}}}
    ]
  }'

Database Management

// Close a collection
db.closeCollection("users");

// Drop a collection
db.dropCollection("users");

// Delete entire database
db.deleteDatabase();

Document Structure

Every document automatically includes:

• _id: Unique identifier (auto-generated)
• createdAt: Creation timestamp (ISO format)
• updatedAt: Last update timestamp (ISO format)

{
  _id: "a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6",
  name: "John Doe",
  email: "[email protected]",
  createdAt: "2026-05-16T10:30:00.000Z",
  updatedAt: "2026-05-16T10:30:00.000Z"
}

Best Practices

1. Use Schema Validation: Define schemas to catch errors early
2. Create Indexes: Index frequently queried fields for better performance
3. Use Transactions: Wrap related operations in transactions for data consistency
4. Validate Input: Validate and sanitize user input before database operations
5. Error Handling: Always handle errors in database operations
6. Regular Backups: Keep backups of your data.json file
7. Use Projections: Only retrieve fields you need with projections

Limitations

• Single-file JSON storage (not suitable for very large datasets)
• No query optimization (all queries scan collections)
• Limited concurrency handling
• Transactions are local to the process instance

License

MIT