xlmdb

Advanced search helper for LMDB with TypeScript support. Make LMDB queries as simple as MongoDB queries!

xlmdb extends LMDB with powerful search capabilities including filtering, deep search, sorting, and prefix matching - all with full TypeScript type safety.

⚠️ Note: This is an experimental package. Use at your own risk. The API may change in future versions.

Features

• 🔍 Advanced filtering - Chain multiple filters with custom logic
• 🌊 Deep search - Find text in nested objects and arrays
• 🎯 Prefix matching - Efficient key-based filtering
• 📊 Custom sorting - Sort results by any criteria
• 🚀 Type-safe - Full TypeScript support with generics
• ⚡ Performance - Built on LMDB's high-performance range queries
• 🎨 Simple API - One function to rule them all: search

Install

bun add xlmdb lmdb
# or
npm install xlmdb lmdb
# or
pnpm add xlmdb lmdb

Quick Start

import { open } from "lmdb";
import { search } from "xlmdb";

// Define your types
type Product = {
  name: string;
  price: number;
  category: string;
  description?: string;
  tags?: string[];
};

// Open database and collection
const root = open({ path: "./data" });
const products = root.openDB<Product, string>({ name: "products" });

// Add some data
await products.put("p1", { 
  name: "Laptop", 
  price: 999, 
  category: "electronics",
  description: "Powerful laptop for gaming"
});

await products.put("p2", { 
  name: "Coffee Maker", 
  price: 89, 
  category: "appliances",
  tags: ["kitchen", "coffee"]
});

// Search with filters, query, and sorting!
const results = search(products, {
  filters: [p => p.price < 1000],           // Filter by price
  query: "coffee",                          // Search in any field
  sort: (a, b) => b.price - a.price,         // Sort by price descending
  limit: 10                                  // Limit results
});

console.log(results);
// [{ key: "p2", value: { name: "Coffee Maker", ... } }]

Full Control

For more control, use LMDB's native API:

import { open } from "lmdb";
import { search } from "xlmdb";

// Open database
const db = open({ path: "./data" });
const products = db.openDB<Product, string>({ name: "products" });

// Use search function
const results = search(products, {
  filters: [p => p.price < 1000],
});

API Reference

search<T, K>(db: Database<T, K>, options?: SearchOptions<T>): Array<{ key: K; value: T }>

Perform advanced search on an LMDB database.

Type Parameters:

• T - Value type (your data structure)
• K - Key type (default: string)

Parameters:

• db - LMDB Database instance
• options - Search options (see below)

Returns: Array of objects with key and value properties

Search Options

interface SearchOptions<T> {
  /** Key prefix to filter results (efficient range query) */
  prefix?: string;
  
  /** Maximum number of results to return */
  limit?: number;
  
  /** Custom sort function applied after filtering */
  sort?: (a: T, b: T) => number;
  
  /** Array of filter functions - all must return true */
  filters?: Array<(value: T, key: string) => boolean>;
  
  /** Query text - searches for text in any string field (recursive) */
  query?: string;
  
  /** Deprecated: Use query instead. Searches for text in any string field (recursive) */
  deepSearch?: string;
  
  /** Additional LMDB range options (reverse, offset, snapshot, transaction, etc) */
  rangeOptions?: Omit<RangeOptions, "start" | "end" | "limit">;
}

Usage Examples

1. Basic Filtering

Filter products by price:

const cheapProducts = search(products, {
  filters: [p => p.price < 50]
});

2. Multiple Filters

Chain multiple filters (all must pass):

const filtered = search(products, {
  filters: [
    p => p.price > 100,
    p => p.price < 1000,
    p => p.category === "electronics",
    p => p.stock > 0
  ]
});

3. Deep Search

Search text in any nested field:

// Finds "gaming" in name, description, tags, or any nested field
const gamingProducts = search(products, {
  query: "gaming"
});

4. Prefix Matching

Efficient key-based filtering:

import { open } from "lmdb";
import { search } from "xlmdb";

// Get all items with keys starting with "user:"
const root = open({ path: "./data" });
const users = root.openDB<User, string>({ name: "users" });
const results = search(users, {
  prefix: "user:"
});

5. Custom Sorting

Sort by any criteria:

const sortedByPrice = search(products, {
  sort: (a, b) => a.price - b.price  // ascending
});

const sortedByName = search(products, {
  sort: (a, b) => a.name.localeCompare(b.name)  // alphabetical
});

6. Limit Results

Get top N results:

const top5 = search(products, {
  sort: (a, b) => b.views - a.views,
  limit: 5
});

7. Combined Features

Combine all features for powerful queries:

const results = search(products, {
  prefix: "p",                                    // Keys starting with "p"
  filters: [                                      // Multiple filters
    p => p.price < 100,
    p => p.category === "electronics"
  ],
  query: "wireless",                             // Search in nested fields
  sort: (a, b) => b.price - a.price,            // Sort by price
  limit: 10                                      // Top 10 results
});

8. Complex Type-Safe Queries

With TypeScript generics, you get full type safety:

import { open } from "lmdb";
import { search } from "xlmdb";

type BlogPost = {
  title: string;
  content: string;
  author: { name: string; email: string };
  tags: string[];
  published: boolean;
  views: number;
};

const root = open({ path: "./data" });
const posts = root.openDB<BlogPost, string>({ name: "posts" });

// Type-safe query with autocomplete!
const recentViews = search(posts, {
  filters: [
    post => post.published,
    post => post.views > 1000
  ],
  query: "TypeScript",
  sort: (a, b) => b.views - a.views,
  limit: 5
});

9. Using with Transactions

import { open } from "lmdb";
import { search } from "xlmdb";

const root = open({ path: "./data" });
const products = root.openDB<Product, string>({ name: "products" });
const tx = root.beginTransaction();
const results = search(products, {
  rangeOptions: { transaction: tx }
});
await tx.commit();

Advanced Usage

Performance Tips

1. Use prefix for key-based queries - Most efficient
2. Apply filters before sorting - Reduces sort work
3. Set reasonable limits - Prevents memory issues
4. Combine with LMDB range options - Use snapshot/transaction for consistency

TypeScript Tips

1. Always specify types when opening databases:

   const root = open({ path: "./data" });
   const products = root.openDB<Product, string>({ name: "products" });

2. Use const assertions for better inference:

   type Product = { name: string; price: number; } as const;

3. Extract filter functions for reusability:

   const affordable = (p: Product) => p.price < 100;
   const inStock = (p: Product) => p.stock > 0;
      
   const results = search(products, {
     filters: [affordable, inStock]
   });
   

Why Use xlmdb?

Without xlmdb (Manual LMDB)

// Tedious manual iteration
const results = [];
for (const { key, value } of products.getRange()) {
  if (value.price < 100 && 
      value.category === "electronics" && 
      value.stock > 0) {
    results.push({ key, value });
  }
}
results.sort((a, b) => a.value.price - b.value.price);
const top5 = results.slice(0, 5);

With xlmdb

// Clean and powerful
const results = search(products, {
  filters: [
    p => p.price < 100,
    p => p.category === "electronics",
    p => p.stock > 0
  ],
  sort: (a, b) => a.price - b.price,
  limit: 5
});

Comparison

| Feature | xlmdb | LMDB Native | MongoDB | |---------|-------|-------------|---------| | Filtering | ✅ | ❌ | ✅ | | Deep Search | ✅ | ❌ | ✅ | | Sorting | ✅ | Manual | ✅ | | Type Safety | ✅ | ✅ | ✅ | | Performance | ⚡ Fast | ⚡⚡ Fastest | ⚡ Fast | | File-based | ✅ | ✅ | ❌ | | No server | ✅ | ✅ | ❌ |

Examples

Try the included examples:

# Run the e-commerce example
bun run example

# Run the animal shelter example
bun run example:shelter

Or check out the example directory for more use cases.

Web UI Explorer

Explore your LMDB databases with a beautiful web interface! xlmdb is being used in local-lmdb-explorer, a local web UI that lets you:

• 🔍 Deep search across all database fields
• 🎯 Advanced filtering with operators (=, !=, >, <, >=, <=)
• 📊 Custom sorting by any field
• 🔎 Automatic database scanner to discover all .mdb files
• 📑 Bookmarks for quick database access
• ⚡ Real-time results as you type

Check it out at: https://github.com/AndrianBalanescu/local-lmdb-explorer

Testing

# Run tests
bun test

# Run tests in watch mode
bun test:watch

All tests pass and examples work out of the box.

Limitations

• Full scans on complex queries - Prefix-based queries are fastest
• No indexing - Filtering happens in-memory after retrieval
• Sync API - Results are returned synchronously (LMDB's design)

Contributing

Contributions welcome! Please:

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests
5. Submit a pull request

License

MIT - See LICENSE file for details.

Acknowledgments

Built on top of LMDB - a fast, memory-mapped database with excellent performance characteristics.