📚 Technical Documentation

MongoDB Pipeline Builder

A type-safe, fluent API for building MongoDB aggregation pipelines

🚀 Overview

mongodb-pipeline-builder is a powerful TypeScript library that simplifies the creation of MongoDB aggregation pipelines. It provides a fluent, type-safe API that makes your aggregation pipelines more readable, maintainable, and less error-prone.

Key Features

✨ Type-Safe - Full TypeScript support with generics for typed responses
📖 Readable - Fluent API that makes complex pipelines easy to understand
🔧 Maintainable - Modular design with reusable helpers
🎯 Complete - Supports all MongoDB aggregation stages and operators
⚡ Efficient - Built-in pagination support with optimized queries
🛡️ Validated - Automatic validation of pipeline stages

Supported Platforms

• ✅ MongoDB Native Driver - db.collection.aggregate()
• ✅ MongoDB Database Commands - db.aggregate()
• ✅ Mongoose - Model.aggregate()

📦 Installation

npm install mongodb-pipeline-builder

yarn add mongodb-pipeline-builder

pnpm add mongodb-pipeline-builder

🎯 Quick Start

Basic Example

import { PipelineBuilder } from 'mongodb-pipeline-builder';
import { ProjectOnlyHelper } from 'mongodb-pipeline-builder/helpers';
import { $Expression, $Equal } from 'mongodb-pipeline-builder/operators';

const pipeline = new PipelineBuilder('users-query')
  .Match($Expression($Equal('$status', 'active')))
  .Project(ProjectOnlyHelper('name', 'email', 'createdAt'))
  .Sort({ createdAt: -1 })
  .Limit(10)
  .build();

// Use with MongoDB
const results = await db.collection('users').aggregate(pipeline).toArray();

// Use with Mongoose
const results = await User.aggregate(pipeline);

With Pagination

import { PipelineBuilder, GetPagingResult } from 'mongodb-pipeline-builder';
import { $Expression, $GreaterThan } from 'mongodb-pipeline-builder/operators';

const pipeline = new PipelineBuilder('paginated-users')
  .Match($Expression($GreaterThan('$age', 18)))
  .Sort({ createdAt: -1 })
  .Paging(20, 1) // 20 items per page, page 1
  .build();

const result = await GetPagingResult(User, pipeline);

console.log(result.GetDocs());           // Document array
console.log(result.GetCount());          // Total count
console.log(result.GetTotalPageNumber()); // Total pages

With Lookups (Joins)

import { PipelineBuilder } from 'mongodb-pipeline-builder';
import { LookupEqualityHelper, Field } from 'mongodb-pipeline-builder/helpers';
import { $ArrayElementAt } from 'mongodb-pipeline-builder/operators';

const pipeline = new PipelineBuilder('users-with-profiles')
  .Lookup(LookupEqualityHelper('profiles', 'profile', 'profileId', '_id'))
  .AddFields(
    Field('profileData', $ArrayElementAt('$profile', 0))
  )
  .Unset('profile')
  .build();

📚 Documentation

Core Concepts

• Getting Started Guide - Detailed introduction and setup
• Migration Guide v3 → v4 - Upgrade from version 3

API Reference

• Pipeline Stages - All MongoDB aggregation stages
• Operators - Aggregation operators ($match, $group, etc.)
• Helpers - Utility functions and stage helpers (Field, ProjectHelper, LookupHelper, etc.)

Examples & Tutorials

• Pagination Examples - Implementing pagination patterns
• Lookup & Join Examples - Working with multiple collections

🔥 What's New in v4

Breaking Changes

PipelineBuilder

• ✨ build() replaces getPipeline()
• 🆕 New stages: ChangeStream, ChangeStreamSplitLargeEvent, Densify, Documents, Fill, ListLocalSessions, ListSampledQueries, ListSearchIndexes, SearchMeta, SetWindowFields, ShardedDataDistribution
• 🆕 Insert() stage for adding custom stages without validation
• ✅ Automatic detection of non-duplicable stages

Helpers

• 🔄 Payload suffix → Helper suffix
• 🏷️ Prefixed with stage name (e.g., LookupEqualityHelper)

Operators

• 🏷️ All operators prefixed with $ (e.g., $Add, $Match)
• 🔄 MapOperator → $Map

Result Methods

• 🎯 GetResult<T>() - For non-paginated queries
  • ✨ GetElement(index | 'last') - New method to get specific document
  • 🚀 Generic type support for typed responses
• 🎯 GetPagingResult<T>() - Exclusively for paginated queries
  • 🚀 Generic type support for typed responses

💡 Real-World Example

import { PipelineBuilder, GetResult } from 'mongodb-pipeline-builder';
import { 
  LookupEqualityHelper, 
  ProjectOnlyHelper, 
  Field 
} from 'mongodb-pipeline-builder/helpers';
import { 
  $Expression, 
  $And,
  $Equal, 
  $GreaterThanEqual,
  $ArrayElementAt 
} from 'mongodb-pipeline-builder/operators';

// Complex query: Active users with their orders
const pipeline = new PipelineBuilder('active-users-with-orders', { debug: true })
  // Filter active users over 18
  .Match($Expression(
    $And(
      $Equal('$status', 'active'),
      $GreaterThanEqual('$age', 18)
    )
  ))
  
  // Join with orders collection
  .Lookup(LookupEqualityHelper('orders', 'orders', '_id', 'userId'))
  
  // Project specific fields
  .Project(ProjectOnlyHelper('name', 'email', 'age'))
  
  // Add computed fields
  .AddFields(
    Field('totalOrders', { $size: '$orders' }),
    Field('lastOrder', $ArrayElementAt('$orders', -1))
  )
  
  // Sort by total orders descending
  .Sort(Field('totalOrders', -1))
  
  // Limit results
  .Limit(50)
  
  .build();

// Execute query with typed response
interface UserWithOrders {
  name: string;
  email: string;
  age: number;
  totalOrders: number;
  lastOrder?: any;
}

const result = await GetResult<UserWithOrders>(User, pipeline);

// Access results
const users = result.GetDocs();        // UserWithOrders[]
const count = result.GetCount();       // number
const firstUser = result.GetElement(0); // UserWithOrders | undefined
const lastUser = result.GetElement('last'); // UserWithOrders | undefined

🧪 Try It Online

→ Try the library on NPM RunKit

📊 Comparison: Before & After

Before (Raw MongoDB)

const pipeline = [
  { 
    $match: { 
      $expr: { 
        $and: [
          { $eq: ["$status", "active"] },
          { $gte: ["$age", 18] }
        ]
      } 
    } 
  },
  {
    $lookup: {
      from: "orders",
      localField: "_id",
      foreignField: "userId",
      as: "orders"
    }
  },
  { 
    $project: { 
      _id: 0, 
      name: 1, 
      email: 1, 
      age: 1,
      totalOrders: { $size: "$orders" },
      lastOrder: { $arrayElemAt: ["$orders", -1] }
    } 
  },
  { $sort: { totalOrders: -1 } },
  { $limit: 50 }
];

After (With Pipeline Builder)

const pipeline = new PipelineBuilder('active-users')
  .Match($Expression($And(
    $Equal('$status', 'active'),
    $GreaterThanEqual('$age', 18)
  )))
  .Lookup(LookupEqualityHelper('orders', 'orders', '_id', 'userId'))
  .Project(
    ProjectOnlyHelper('name', 'email', 'age'),
    Field('totalOrders', { $size: '$orders' }),
    Field('lastOrder', $ArrayElementAt('$orders', -1))
  )
  .Sort(Field('totalOrders', -1))
  .Limit(50)
  .build();

🔍 Working with Results

GetResult - For Non-Paginated Queries

Use when your pipeline does not include the Paging stage:

import { GetResult } from 'mongodb-pipeline-builder';

interface User {
  name: string;
  email: string;
  age: number;
}

const pipeline = new PipelineBuilder('users')
  .Match($Expression($Equal('$status', 'active')))
  .build();

const result = await GetResult<User>(User, pipeline);

const users = result.GetDocs();           // User[] - all documents  
const count = result.GetCount();          // number - total count
const firstUser = result.GetElement(0);   // User | undefined
const lastUser = result.GetElement('last'); // User | undefined

GetPagingResult - For Paginated Queries

Use exclusively when your pipeline includes the Paging stage:

import { GetPagingResult } from 'mongodb-pipeline-builder';

const pipeline = new PipelineBuilder('users')
  .Match($Expression($Equal('$status', 'active')))
  .Paging(20, 1) // Required for GetPagingResult
  .build();

const result = await GetPagingResult<User>(User, pipeline);

const users = result.GetDocs();           // User[] - current page
const totalCount = result.GetCount();     // number - total documents
const totalPages = result.GetTotalPageNumber(); // number - total pages

📖 Learn More: See Getting Started Guide for detailed examples

📚 API Reference Quick Links

Pipeline Stages

All MongoDB aggregation stages are supported. See complete reference.

Common Stages:

• Match() - Filter documents
• Project() - Select/transform fields
• Group() - Group and aggregate
• Sort() - Sort documents
• Limit() - Limit results
• Lookup() - Join collections
• Unwind() - Deconstruct arrays
• AddFields() - Add computed fields
• Paging() - Add pagination

Operators

100+ MongoDB operators supported. See complete reference.

Common Operators:

• Comparison: $Equal, $GreaterThan, $LessThan
• Logical: $And, $Or, $Not
• Arithmetic: $Add, $Subtract, $Multiply, $Divide
• Array: $Size, $Filter, $Map, $ArrayElementAt
• String: $Concat, $ToLower, $ToUpper, $Split
• Date: $DateAdd, $DateSubtract, $DateDifference
• Aggregation: $Sum, $Average, $Min, $Max

Helpers

20+ helper functions for common patterns. See complete list in stages.md.

Common Helpers:

• Field(name, value) - Universal field helper
• ProjectOnlyHelper(...fields) - Include specific fields
• LookupEqualityHelper(...) - Simple joins
• BucketHelper(...) - Categorize documents
• SearchHelper(...) - Full-text search

🎓 Quick Examples

Example 1: Simple Query

const pipeline = new PipelineBuilder('active-users')
  .Match($Expression($Equal('$status', 'active')))
  .Project(ProjectOnlyHelper('name', 'email'))
  .Sort({ createdAt: -1 })
  .Limit(10)
  .build();

const result = await GetResult<User>(User, pipeline);

Example 2: With Pagination

const pipeline = new PipelineBuilder('paginated')
  .Match($Expression($GreaterThan('$price', 100)))
  .Sort({ price: -1 })
  .Paging(20, 1)
  .build();

const result = await GetPagingResult<Product>(Product, pipeline);
console.log(`Page 1 of ${result.GetTotalPageNumber()}`);

Example 3: With Joins

const pipeline = new PipelineBuilder('users-with-orders')
  .Lookup(LookupEqualityHelper('orders', 'orders', '_id', 'userId'))
  .AddFields(
    Field('orderCount', $Size('$orders')),
    Field('lastOrder', $ArrayElementAt('$orders', -1))
  )
  .Match($Expression($GreaterThan('$orderCount', 0)))
  .build();

Example 4: Complex Aggregation

const pipeline = new PipelineBuilder('sales-by-category')
  .Match($Expression($Equal('$year', 2024)))
  .Group({
    _id: '$category',
    totalSales: $Sum('$amount'),
    averagePrice: $Average('$price'),
    count: $Sum(1)
  })
  .Sort({ totalSales: -1 })
  .Limit(10)
  .build();

❓ FAQ

• Use GetResult for queries without the Paging stage
• Use GetPagingResult exclusively when using the Paging stage

The Paging stage wraps your pipeline in $facet for efficient pagination.

Yes! Use the Insert stage:

builder.Insert({ $myCustomStage: { field: 'value' } })

Useful for new MongoDB stages or custom implementations.

Yes! Works with:

• MongoDB Native Driver: db.collection.aggregate()
• Mongoose: Model.aggregate()
• MongoDB Database Commands: db.aggregate()

Yes! All stages through MongoDB 7.0+ including $densify, $fill, $setWindowFields, $searchMeta, and more.

Use generics:

interface User { name: string; email: string; }
const result = await GetResult<User>(User, pipeline);
const users: User[] = result.GetDocs(); // Fully typed!

Yes! Chain as many as needed:

builder
  .Lookup(LookupEqualityHelper('profiles', 'profile', '_id', 'userId'))
  .Lookup(LookupEqualityHelper('orders', 'orders', '_id', 'userId'))
  .Lookup(LookupEqualityHelper('subscriptions', 'sub', '_id', 'userId'))

See Lookup Examples.

📈 Performance Tips

1. Index your fields - Create indexes on $match and $sort fields
2. Match early - Apply $match as early as possible
3. Project early - Remove unnecessary fields to reduce memory
4. Limit lookups - Use pipeline syntax to limit joined documents
5. Use covered queries - Query only indexed fields when possible

📖 Learn More: Pagination Examples

🆘 Support

Getting Help

• 📖 Documentation
• 🐛 Issue Tracker
• 💬 MongoDB Docs

Reporting Issues

Include:

• Package version
• Node.js and MongoDB versions
• Code snippet
• Expected vs actual behavior

📜 Changelog

See CHANGELOG.md for version history.

Made with ❤️ by Mickaël NODANCHE