ts-migrate-mongoose

A node/typescript based migration framework for mongoose

Motivation

ts-migrate-mongoose is a migration framework for projects which are already using mongoose

Features

• [x] Stores migration state in MongoDB
• [x] Flexibility in configuration migrate.json or migrate.ts or .env and/or .env.local
• [x] Use mongoose models when running migrations
• [x] Use async/await in migrations
• [x] Run migrations from the CLI
• [x] Run migrations programmatically
• [x] Prune old migrations, and sync new migrations
• [x] Create custom templates for migrations
• [x] Supports ~~ESM~~ (not yet) and CommonJS

Example

How to use it with:

• Express: ts-express-swc, ts-express-esbuild
• Nest: ts-express-nest

Installation

• Locally inside your project

yarn add ts-migrate-mongoose
npm install ts-migrate-mongoose
pnpm add ts-migrate-mongoose

• Install it globally

yarn global add ts-migrate-mongoose
npm install -g ts-migrate-mongoose
pnpm add -g ts-migrate-mongoose

Migrations and alias imports

If you are using alias imports in your project, you can use tsconfig.json paths to resolve them for you project.
But ts-migrate-mongoose uses swc to compile the migrations internally, so you also need to add .swcrc file to project root
Starting from "@swc/core": "1.3.74", you need to use target or env not both, in example bellow we use "target": "es2021"

{
  "jsc": {
    "parser": {
      "syntax": "typescript",
      "decorators": true,
      "dynamicImport": true
    },
    "target": "es2021",
    "keepClassNames": true,
    "loose": true,
    // Important part bellow, copy it from your tsconfig.json
    "baseUrl": ".",
    "paths": {
      "@/*": ["src/*"]
    }
  },
  "module": {
    "type": "commonjs"
  },
  "sourceMaps": true
}

Now your migrations will be compiled with swc and you can use alias imports in your migrations.

Configuration

If you don't want to provide -d or --uri flag in CLI or Programmatic mode, you can configure it.
Create a migrate.json or migrate.ts or .env file in the root of your project:

• migrate.json

{
  "uri": "mongodb://localhost/my-db",
  "collection": "migrations",
  "migrationsPath": "./migrations",
  "templatePath": "./migrations/template.ts",
  "autosync": false
}

• migrate.ts

export default {
  uri: "mongodb://localhost/my-db",
  collection: "migrations",
  migrationsPath: "./migrations",
  templatePath: "./migrations/template.ts",
  autosync: false,
};

• .env

MIGRATE_MONGO_URI=mongodb://localhost/my-db
MIGRATE_MONGO_COLLECTION=migrations
MIGRATE_CONFIG_PATH=./migrate
MIGRATE_MIGRATIONS_PATH=./migrations
MIGRATE_TEMPLATE_PATH=./migrations/template.ts
MIGRATE_AUTOSYNC=false
# or 
migrateMongoUri=mongodb://localhost/my-db
migrateMongoCollection=migrations
migrateConfigPath=./migrate
migrateMigrationsPath=./migrations
migrateTemplatePath=./migrations/template.ts
migrateAutosync=false

| Config file | .env / export | Default | Required | Description | | -------------------- | ------------------------ | ------------ | -------- | ------------------------------------------------ | | uri | MIGRATE_MONGO_URI | - | Yes | mongo connection string | | collection | MIGRATE_MONGO_COLLECTION | migrations | No | collection name to use for the migrations | | configPath | MIGRATE_CONFIG_PATH | ./migrate | No | path to the config file | | migrationsPath | MIGRATE_MIGRATIONS_PATH | ./migrations | No | path to the migration files | | templatePath | MIGRATE_TEMPLATE_PATH | - | No | template file to use when creating a migration | | autosync | MIGRATE_AUTOSYNC | false | No | automatically sync new migrations without prompt |

Getting started with the CLI

yarn migrate help
npx migrate help
pnpm migrate help

CLI migration tool for mongoose

Options:
  -f, --config-path <path>      path to the config file (default: "migrate")
  -d, --uri <string>            mongo connection string
  -c, --collection <string>     collection name to use for the migrations (default: "migrations")
  -a, --autosync <boolean>      automatically sync new migrations without prompt (default: false)
  -m, --migrations-path <path>  path to the migration files (default: "./migrations")
  -t, --template-path <path>    template file to use when creating a migration
  -h, --help                    display help for command

Commands:
  list                          list all migrations
  create <migration-name>       create a new migration file
  up [migration-name]           run all migrations or a specific migration if name provided
  down <migration-name>         roll back migrations down to given name
  prune                         delete extraneous migrations from migration folder or database
  help [command]                display help for command

• Examples yarn

yarn migrate list -d mongodb://localhost/my-db
yarn migrate create add_users -d mongodb://localhost/my-db
yarn migrate up add_user -d mongodb://localhost/my-db
yarn migrate down delete_names -d mongodb://localhost/my-db
yarn migrate down # will rollback one migration
yarn migrate prune -d mongodb://localhost/my-db
yarn migrate list --config settings.json

• Examples npm

npm run migrate list -d mongodb://localhost/my-db
npm run migrate create add_users -d mongodb://localhost/my-db
npm run migrate up add_user -d mongodb://localhost/my-db
npm run migrate down delete_names -d mongodb://localhost/my-db
npm run migrate down # will rollback one migration
npm run migrate prune -d mongodb://localhost/my-db
npm run migrate list --config settings.json

• Examples pnpm

pnpm migrate list -d mongodb://localhost/my-db
pnpm migrate create add_users -d mongodb://localhost/my-db
pnpm migrate up add_user -d mongodb://localhost/my-db
pnpm migrate down delete_names -d mongodb://localhost/my-db
pnpm migrate down # will rollback one migration
pnpm migrate prune -d mongodb://localhost/my-db
pnpm migrate list --config settings.json

Options override order

Note that options are overridden in the following order:

• Command line args > Env vars > Config file

Migration files

This example demonstrates how you can create a migration file using the CLI
By default, ts-migrate-mongoose assumes your migration folder exists (if it does not it will create one for you)

Here's an example of a migration created using:

yarn migrate create first-migration-demo
npx migrate create first-migration-demo
pnpm migrate create first-migration-demo

Executing the above command will create a migration file in the ./migrations folder with the following content:

• 1673525773572-first-migration-demo.ts

// Import your models here

export async function up (): Promise<void> {
  // Write migration here
}

export async function down (): Promise<void> {
  // Write migration here
}

Using mongoose models in your migrations

As long as you can import the references to your models you can use them in migrations
Below is an example of a typical setup in a mongoose project:

• models/User.ts - defines the User model

import { Schema, model, models } from 'mongoose'

interface IUser {
  firstName: string
  lastName?: string
}

const UserSchema = new Schema<IUser>({
  firstName: {
    type: String,
    required: true
  },
  lastName: {
    type: String
  }
})

export default models.User ?? model<IUser>('User', UserSchema)

• models/index.ts - ensures that all models are exported and you establish a connection to the database

import mongoose from 'mongoose'
import mongooseOptions from '../options/mongoose'

import User from './User'

const getModels = async () => {
  // In case you using mongoose 6
  // https://mongoosejs.com/docs/guide.html#strictQuery
  mongoose.set('strictQuery', false)

  // Ensure connection is open so we can run migrations
  await mongoose.connect(process.env.MIGRATE_MONGO_URI ?? 'mongodb://localhost/my-db', mongooseOptions)

  // Return models that will be used in migration methods
  return {
    mongoose,
    User
  }
}

export default getModels

• 1673525773572-first-migration-demo.ts - your migration file

import getModels from '../models'

export async function up () {
  const { User } = await getModels()
  // Write migration here
  await User.create([
    {
      firstName: 'John',
      lastName: 'Doe'
    },
    {
      firstName: 'Jane',
      lastName: 'Doe'
    }
  ])
}

export async function down () {
  const { User } = await getModels()
  // Write migration here
  await User.deleteMany({ firstName: { $in: ['Jane', 'John'] } }).exec()
}

Notes

• Currently, the -d or --uri must include the database to use for migrations in the uri.
• Example: -d mongodb://localhost:27017/development
• If you don't want to pass it in every time feel free to use migrate.ts or migrate.json config file or an environment variable
• Feel Free to check out the /examples folder in the project to get a better idea of usage in Programmatic and CLI mode

Check my other projects

• ts-patch-mongoose - Patch history & events plugin for mongoose
• ts-cache-mongoose - Cache plugin for mongoose Queries and Aggregate (in-memory, redis)