mingo

MongoDB query language for in-memory objects

Install

$ npm install mingo

Features

• Dot notation selectors. <array>.<index> and <document>.<field>.
• Query and Projection.
• Aggregation pipeline with support for;
  • Accumulators
  • Expressions
  • Window operators
  • Variables; $$ROOT, $$CURRENT, $$DESCEND, $$PRUNE, $$KEEP, $$REMOVE, $$NOW
• Update support for collections and objects. See Updating Documents.
• Lazy data processing pipeline.
• Custom type value equality when toString is implemented.

API

Below are the most commonly used objects exported in the default module.

• Functions: find, aggregate, update, updateMany, updateOne.
• Classes: Query, Aggregator

See here for full module documentation. For information on how to use specific operators refer to official MongoDB website.

Use the playground to try out this library in a fully functional environment.

Usage

import * as mingo from "mingo";

const result = find(
  [
    { name: 'Alice', age: 30 },
    { name: 'Bob', age: 21 },
    { name: 'Charlie', age: 25 },
  ],
  { age: { $gte: 25 } }
).all()

console.log(result)
/*
[
  { "name": "Alice", "age": 30 },
  { "name": "Charlie", "age": 25 }
]
*/

NB: Objects imported through the default entry point automatically add all operators into their context.

Loading Operators

To load only specific operators in your environment, you should import objects from their base modules such as mingo/{core,query,aggregator,updater}, and then import and register operators into a custom Context to configure in your options. This is necessary if you use a bundler and want to exclude unused objects during tree-shaking.

Operators loaded into a Context are immutable. Registering the same operator name is a no-op and does not throw an error. This prevents unintentional errors.

// In this example the only operators available will be $match, $count, and $gt.
// Attempting to use any other operator will throw an error.
import { Context } from "mingo/core";
import { Aggregator } from "mingo/aggregator";
import { $match, $count } from "mingo/operators/pipeline";
import { $gt } from "mingo/operators/expression/comparison";

// creates a context with only operators needed for execution.
const context = Context.init({
  pipeline: { $count, $match },
  expression: { $gt }
});

const agg = new Aggregator(
  [{ $match: { score: { $gt: 80 } } }, { $count: "passing_scores" }],
  { context } // pass context as part of options
);

const result = agg.run([
  { _id: 1, score: 10 },
  { _id: 2, score: 60 },
  { _id: 3, score: 100 }
]);

Using Query for predicate tests

// Query imported from default entry point. Automatically loads all operators into context.
import { Query } from "mingo";

// create a query with criteria
// find all grades for homework with score >= 50
let query = new Query({
  type: "homework",
  score: { $gte: 50 }
});

// test if an object matches query
query.test(doc); // returns boolean if `doc` matches criteria.

Searching and Filtering

// Query imported from default entry point. Automatically loads all operators into context.
import { Query } from "mingo";

const query = new Query({ score: { $gt: 10 } });

// filter some `collection` with find()
const cursor = query.find(collection);

// sort, skip and limit by chaining
cursor.sort({ student_id: 1, score: -1 }).skip(100).limit(100);

for (const value of cursor) {
  console.log(value); // print each match
}

Using $jsonSchema operator

To use the $jsonSchema operator, you must register your own JsonSchemaValidator in the options. No default implementation is provided out of the box so users can use a library with their preferred schema format.

The example below uses Ajv to implement schema validation.

import mingo from "mingo"
import type { AnyObject, JsonSchemaValidator } from "mingo/types"
import Ajv, { Schema } from "ajv"

const jsonSchemaValidator: JsonSchemaValidator = (s: AnyObject) => {
  const ajv = new Ajv();
  const v = ajv.compile(s as Schema);
  return (o: AnyObject) => (v(o) ? true : false);
};

const schema = {
  type: "object",
  required: ["item", "qty", "instock"],
  properties: {
    item: { type: "string" },
    qty: { type: "integer" },
    size: {
      type: "object",
      required: ["uom"],
      properties: {
        uom: { type: "string" },
        h: { type: "number" },
        w: { type: "number" },
      },
    },
    instock: { type: "boolean" },
  },
};

// queries documents using schema validation
mingo.find(docs, { $jsonSchema: schema }, {}, { jsonSchemaValidator }).all();

NB: An error is thrown when the $jsonSchema operator is used without a the jsonSchemaValidator configured.

Aggregation

import { Aggregator } from "mingo/aggregator";
import { Context } from "mingo/core";
import { $group } from "mingo/operators/pipeline/group";
import { $match } from "mingo/operators/pipeline/match";
import { $sort } from "mingo/operators/pipeline/sort";
import { $min } from "mingo/operators/accumulator/min";

// ensure the required operators are preloaded prior to using them.
const context = Context.init({
  pipeline: { $group, $match, $sort },
  accumulator: { $min }
});

let agg = new Aggregator(
  [
    { $match: { type: "homework" } },
    { $group: { _id: "$student_id", score: { $min: "$score" } } },
    { $sort: { _id: 1, score: 1 } }
  ],
  { context }
);

// return an iterator for streaming results
let result = agg.stream(collection).collect(); // same as `agg.run(collection)`

Custom Operators

Custom operators can be registered using a Context object via the context option which is the recommended way since 6.4.2. Context provides a container for operators that the execution engine will use to process queries.

Operators must conform to the signatures of their types. See mingo/core module for types.

Example: Add custom $between query operator

import { Query } from "mingo/query"
import { Context } from "mingo/core"
import { resolve } from "mingo/util"

// this example creates a query operator that checks if a value is between a boundary.
const $between = (selector, args, options) => {
  return obj => {
    const value = resolve(obj, selector);
    return value >= args[0] && value <= args[1];
  };
};

const context = Context.init().addQueryOps({ $between })
// pass the context to options
const q = new Query({ a: { $between: [5, 10] } }, { context })

// a test collection
const collection = [
  { a: 1, b: 1 },
  { a: 7, b: 1 },
  { a: 10, b: 6 },
  { a: 20, b: 10 }
];

const result = q.find(collection).all();

console.log(result); // output => [ { a: 7, b: 1 }, { a: 10, b: 6 } ]

Updating Documents

The updateOne and updateMany functions can be used to update collections. These work similarly to the methods of the same names on MongoDB collections. These functions operate on an input collection and may modify objects within the collection in-place, or replace them completely. They also allow using supported pipeline operators as modifiers.

For updating a single object reference use the update function.

Example: Modify object with update()

import { update } from "mingo";

const obj = {
  firstName: "John",
  lastName: "Wick",
  age: 40,
  friends: ["Scooby", "Shagy", "Fred"]
};

// returns array of modified paths if value changed.
update(obj, { $set: { firstName: "Bob", lastName: "Doe" } }); // ["firstName", "lastName"]

// update nested values.
update(obj, { $pop: { friends: 1 } }); // ["friends"] => friends: ["Scooby", "Shagy"]
// update nested value path
update(obj, { $unset: { "friends.1": "" } }); // ["friends.1"] => friends: ["Scooby", null]
// update with condition
update(obj, { $set: { "friends.$[e]": "Velma" } }, [{ e: null }]); // ["friends"] => friends: ["Scooby", "Velma"]
// empty array returned if value has not changed.
update(obj, { $set: { firstName: "Bob" } }); // [] => no change to object.

Options

Query and aggregation operations can be configured with options to enabled different features or customize how documents are processed. Some options are only relevant to specific operators and need not be specified if not required.

| Name | Default | Description| | ------------------- | ------- | ---------- | | collation | none | Collation specification for string sorting operations. See Intl.Collator | | collectionResolver | none | Function to resolve strings to arrays for use with operators that reference other collections such as; $lookup, $out and $merge.Expects: (string) => AnyObject[]. | | context | none | An object that defines which operators should be used.This option allow users to load only desired operators or register custom operators which need not be available globally. | | hashFunction | default | Custom hash function to replace the default based on "Effective Java" hashCode.Expects: (Any) => number. | | idKey | "_id" | The key that is used to lookup the ID value of a document. | | jsonSchemaValidator | none | JSON schema validator to use for the $jsonSchema operator.Expects: (schema: AnyObject) => (document: AnyObject) => boolean.The $jsonSchema operation would fail if a validator is not provided. | | processingMode | CLONE_OFF | Determines how inputs are to be modified or not during processing. | | scriptEnabled | true | Enable or disable using custom script execution.When disabled, operators that execute custom code are disallowed such as; $where, $accumulator, and $function. | | useStrictMode | true | Enforces strict MongoDB compatibility. When disabled the behaviour changes as follows.$elemMatch returns all matching documents instead of only the first.Empty string "" is coerced to false during boolean checks.$type returns JS native type names. MongoDBJS"missing""undefined""bool""boolean""int"|"long"|"double""number""regex""regexp" | | variables | none | Global variables to pass to all operators. |

Distribution

The library provides 3 distributions on NPM.

1. A browser bundle that exposes a global mingo object. See unpkg.
2. A CommonJS module under ./cjs when loaded using require or in NodeJS environments.
3. An ESM module under ./esm which is the default export. Can load as module from esm.run.

Supporting both CJS and ESM modules makes this library subject to the dual package hazard. In backend environments, be consistent with the module loading format to avoid surprises and subtle errors. You can avoid this by loading from only the default exports to get all operators, or creating a custom bundle with your favorite bundler.

Examples for using in Browser

Load as global object

<script src="https://unpkg.com/mingo/dist/mingo.min.js"></script>
<script>
  // global 'mingo' module available in scope
  console.log((new mingo.Query({a:5})).test({a:10})) // false
</script>

Load as ESM module

<script type="module">
  import * as mingo from "https://esm.run/mingo/esm/index.js";
  // use objects on 'mingo' module with full operator support.
  console.log((new mingo.Query({a:5})).test({a:10})) // false
</script>

Differences from MongoDB

Below is a description of how this library differs from the full MongoDB query engine.

1. Selectors using <array>.<index> are supported for filter and projection expressions to access specific array elements.
2. No support for server specific types, operators, or features dependent on persistence (e.g. merge option for $accumulator);
3. No support for geometry operators.
4. $merge operator enforces unique constraint during processing.
5. Function evaluation operators; $where, $function, and $accumulator, do not accept strings as the function body.

Benefits

• Declarative data driven API usable on both frontend and backend and suitable for serialization.
• Provides an alternative to writing custom code for transforming objects.
• Validate MongoDB queries without running a server. See playground.
• Well documented. MongoDB query language is among the best available and has great documentation.

Contributing

• Squash changes into one commit.
• Run npm test to build and run unit tests.
• Submit pull request.

To validate correct behaviour and semantics of operators, you may also test against mongoplayground.net. Credit to the author @feliixx.

A big thank you to all users and CONTRIBUTORS of this library.

License

MIT