couchdb-migrate

Schema-first migration tool for CouchDB with automatic migration generation, blue-green deployments, and rollback support.

Version: 0.1.0

Table of Contents

1. Overview
2. Why couchdb-migrate?
3. Installation
4. Quick Start
5. Core Concepts
6. Schema Management
7. Migration Generation
8. Migration Execution
9. Manual Migrations
10. CLI Commands Reference
11. Real-World Examples
12. Production Deployment
13. Best Practices
14. Troubleshooting
15. API Reference
16. Contributing

Overview

What is couchdb-migrate?

couchdb-migrate is a schema-first migration tool for CouchDB that brings structured database evolution to document databases. While CouchDB is schema-less at the storage level, applications always have implicit schemas. This tool makes those schemas explicit, versioned, and manageable.

Key Features

• ✅ Schema-First Development - Define document structures using JSON Schema
• ✅ Automatic Migration Generation - Detect schema changes and generate migrations automatically
• ✅ 27 Change Types Detected - Comprehensive detection of field, index, view, and validation changes
• ✅ Breaking Change Detection - Identifies changes that require manual intervention
• ✅ Blue-Green Deployments - Zero-downtime design document updates with view building
• ✅ State Tracking - Reliable migration state management in CouchDB
• ✅ Distributed Locking - Prevents concurrent migrations with heartbeat mechanism
• ✅ Rollback Support - Safe rollback with down() functions
• ✅ Checksum Validation - Ensures migration integrity
• ✅ Manual Migrations - Write custom migration logic when needed
• ✅ TypeScript First - Full type safety throughout

Why couchdb-migrate?

The Problem

Without explicit schema management:

• ❌ Schema evolution happens ad-hoc in application code
• ❌ Different document versions coexist without clear migration paths
• ❌ Breaking changes can crash production applications
• ❌ No audit trail of schema changes
• ❌ Difficult to coordinate schema changes across teams
• ❌ View rebuilds cause downtime

The Solution

With couchdb-migrate:

• ✅ Schema evolution is explicit and versioned
• ✅ Migrations are generated automatically from schema changes
• ✅ Breaking changes are detected and flagged
• ✅ Complete audit trail in version control
• ✅ Team coordination through code review
• ✅ Zero-downtime view updates with blue-green deployments

Installation

Global Installation (Recommended)

npm install -g couchdb-migrate

Project Dependency

npm install --save-dev couchdb-migrate

Requirements

• Node.js 18+
• CouchDB 2.x or 3.x
• TypeScript 5+ (optional, for type generation)

Quick Start

1. Initialize Project

mkdir my-couchdb-project
cd my-couchdb-project
couchdb-migrate init

This creates:

• schemas/ - Your JSON Schema definitions
• migrations/ - Generated and manual migrations
• generated/ - Snapshot and generated files (gitignored)
• couchdb-migrate.config.ts - Configuration file

2. Configure Database Connection

Edit couchdb-migrate.config.ts:

export default {
  connection: {
    url: 'http://admin:password@localhost:5984',
    database: 'myapp'
  },
  paths: {
    schemas: './schemas',
    migrations: './migrations',
    generated: './generated'
  }
}

3. Create a Schema

couchdb-migrate schema:create user

Edit schemas/user.schema.json:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://example.com/schemas/user.schema.json",
  "title": "User",
  "type": "object",
  "required": ["_id", "type", "email"],
  "properties": {
    "_id": {
      "type": "string",
      "pattern": "^user:[a-z0-9-]+$",
      "description": "Unique identifier"
    },
    "type": {
      "type": "string",
      "const": "user",
      "description": "Document type discriminator"
    },
    "email": {
      "type": "string",
      "format": "email",
      "description": "User email address"
    },
    "name": {
      "type": "string",
      "description": "User full name"
    },
    "createdAt": {
      "type": "string",
      "format": "date-time"
    },
    "updatedAt": {
      "type": "string",
      "format": "date-time"
    }
  },
  "x-couchdb": {
    "type": "user",
    "version": 1,
    "indexes": [
      {
        "name": "by-email",
        "fields": ["email"],
        "unique": true
      }
    ],
    "views": [],
    "validation": {
      "immutableFields": ["_id", "type", "createdAt"]
    }
  }
}

4. Generate Migration

couchdb-migrate generate

Output:

✓ Loaded 1 schema(s)
✓ No snapshot found - creating initial migration

Schema Changes:
  SCHEMA_ADDED user [BREAKING]

Breaking Changes: 1

✓ Generated migration: migrations/20260125120000-schema-changes.ts
✓ Updated schema snapshot

5. Review Generated Migration

cat migrations/20260125120000-schema-changes.ts

6. Run Migration

couchdb-migrate up

Output:

Running migration: 20260125120000-schema-changes
✓ Migration completed in 45ms

Applied Migrations: 1
Pending Migrations: 0

7. Verify Status

couchdb-migrate status

Core Concepts

1. Schemas

Schemas are JSON Schema files (Draft 2020-12) that define document structure:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://example.com/schemas/post.schema.json",
  "type": "object",
  "properties": {
    "_id": { "type": "string" },
    "title": { "type": "string", "minLength": 1, "maxLength": 200 },
    "content": { "type": "string" },
    "author": { "type": "string" },
    "published": { "type": "boolean", "default": false }
  },
  "required": ["_id", "type", "title", "author"],
  "x-couchdb": {
    "type": "post",
    "version": 1,
    "indexes": [
      {
        "name": "by-author",
        "fields": ["author", "createdAt"]
      }
    ]
  }
}

CouchDB Extensions (x-couchdb)

• type: Document type (used for queries)
• version: Schema version number
• indexes: Mango index definitions
• views: Map/reduce view definitions
• validation: Validation rules
• security: Access control settings

2. Migrations

Migrations are TypeScript/JavaScript files that define database changes:

import type { Migration, MigrationContext } from 'couchdb-migrate';

const migration: Migration = {
  id: '20260125120000-add-email-field',
  name: 'add-email-field',
  type: 'schema-update',

  async up(context: MigrationContext): Promise<void> {
    const { db, helpers, logger } = context;

    logger.info('Adding email field to users');

    await helpers.transformDocuments({
      selector: { type: 'user' },
      batchSize: 100,
      transform: (doc) => {
        if (!doc.email) {
          doc.email = `${doc._id}@example.com`;
        }
        return doc;
      }
    });
  },

  async down(context: MigrationContext): Promise<void> {
    // Rollback logic
  }
};

export default migration;

3. Snapshots

Snapshots track the current state of your schemas:

{
  "version": "2.0",
  "timestamp": "2026-01-25T12:00:00.000Z",
  "schemas": {
    "https://example.com/schemas/user.schema.json": {
      "id": "https://example.com/schemas/user.schema.json",
      "path": "schemas/user.schema.json",
      "checksum": "abc123...",
      "couchdb": {
        "type": "user",
        "version": 1
      }
    }
  },
  "checksum": "def456..."
}

Checksums enable fast change detection by comparing file hashes instead of deep object comparisons.

4. State Management

Migration state is stored in CouchDB as a _local document:

{
  "_id": "_local/couchdb-migrate",
  "version": "2.0",
  "applied": [
    {
      "id": "20260125120000-schema-changes",
      "name": "schema-changes",
      "type": "schema-update",
      "appliedAt": "2026-01-25T12:00:00.000Z",
      "checksum": "abc123...",
      "executionTimeMs": 45,
      "appliedBy": "admin"
    }
  ],
  "designDocVersions": {},
  "schemaVersions": {},
  "lastRunAt": "2026-01-25T12:00:00.000Z"
}

Schema Management

Creating Schemas

# Create a new schema
couchdb-migrate schema:create product

# Create with custom type name
couchdb-migrate schema:create product --type inventory-item

Validating Schemas

couchdb-migrate schema:validate

Output:

✓ Validating schemas

✓ https://example.com/schemas/user.schema.json: Valid
✓ https://example.com/schemas/post.schema.json: Valid
  ⚠ properties.status: Consider adding enum constraint

✓ All 2 schema(s) are valid
⚠ Found 1 warning(s)

Checking Schema Changes

couchdb-migrate schema:diff

Output:

Modified Schemas:
  ~ user.schema.json

Summary:
  New: 0
  Modified: 1
  Removed: 0

Schema Evolution Example

// v1 - Initial schema
{
  "properties": {
    "email": { "type": "string" }
  },
  "x-couchdb": { "version": 1 }
}

// v2 - Add email verification
{
  "properties": {
    "email": { "type": "string", "format": "email" },
    "emailVerified": { "type": "boolean", "default": false },
    "emailVerifiedAt": { "type": "string", "format": "date-time" }
  },
  "required": ["email", "emailVerified"],
  "x-couchdb": { "version": 2 }
}

Migration Generation

Automatic Generation

couchdb-migrate generate

This:

1. Loads current schemas
2. Compares with snapshot
3. Detects changes (27 types)
4. Classifies breaking vs non-breaking
5. Generates migration file
6. Updates snapshot

Change Types Detected

Field Changes:

• Field added
• Field removed
• Field type changed
• Field required status changed
• Field made nullable
• Field enum values changed

Constraint Changes:

• minLength/maxLength changed
• minimum/maximum changed
• Pattern changed
• Format changed

Index Changes:

• Index added
• Index removed
• Index modified
• Unique constraint added/removed

View Changes:

• View added
• View removed
• Map/reduce function changed

Other Changes:

• Validation rules changed
• Security settings changed
• Schema type changed

Breaking Changes

Breaking changes require manual review:

⚠ Found 2 breaking change(s)

Breaking changes:
  1. Field 'email' is now required (was optional)
  2. Field 'age' type changed from string to number

You must add manual migration code to handle these changes.

Generation Options

# Dry run (show what would be generated)
couchdb-migrate generate --dry-run

# Force generation even if no changes
couchdb-migrate generate --force

# Skip interactive prompts (use defaults)
couchdb-migrate generate --no-prompt

Migration Execution

Running Migrations

# Run all pending migrations
couchdb-migrate up

# Run to a specific migration
couchdb-migrate up --to 20260125120000

# Dry run (show what would run)
couchdb-migrate up --dry-run

# Skip checksum validation
couchdb-migrate up --force

Viewing Status

# Basic status
couchdb-migrate status

# Verbose mode (show details)
couchdb-migrate status --verbose

# JSON output
couchdb-migrate status --json

Rolling Back

# Rollback last migration
couchdb-migrate down

# Rollback multiple migrations
couchdb-migrate down --steps 3

# Rollback to specific migration
couchdb-migrate down --to 20260120120000

# Dry run
couchdb-migrate down --dry-run

# Skip confirmation
couchdb-migrate down --force

Blue-Green Deployment

Design documents are deployed using a blue-green strategy for zero downtime:

await helpers.deployDesignDoc({
  _id: '_design/myapp',
  views: {
    byDate: {
      map: 'function(doc) { emit(doc.createdAt, null); }'
    }
  }
});

Process:

1. Deploy new version: _design/myapp@2
2. Build views in background (old version still serving)
3. Wait for view build completion
4. Atomic switch: Update _design/myapp pointer
5. Keep old version for instant rollback
6. Cleanup after grace period (default 60s)

Benefits:

• Zero downtime
• No query failures during deployment
• Instant rollback capability
• Progress monitoring

Manual Migrations

Creating Manual Migrations

couchdb-migrate create add-admin-user

Migration Template

import type { Migration, MigrationContext } from 'couchdb-migrate';

const migration: Migration = {
  id: '20260125150000-add-admin-user',
  name: 'add-admin-user',
  type: 'data',

  async up(context: MigrationContext): Promise<void> {
    const { db, helpers, logger } = context;

    logger.info('Creating admin user');

    const adminUser = {
      _id: 'user:admin',
      type: 'user',
      email: '[email protected]',
      role: 'admin',
      createdAt: new Date().toISOString()
    };

    await db.insert(adminUser);
    logger.info('Admin user created');
  },

  async down(context: MigrationContext): Promise<void> {
    const { db, logger } = context;

    logger.info('Removing admin user');
    const doc = await db.get('user:admin');
    await db.destroy(doc._id, doc._rev);
    logger.info('Admin user removed');
  }
};

export default migration;

Migration Helpers

// Design document deployment (blue-green)
await helpers.deployDesignDoc({
  _id: '_design/myapp',
  views: { /* ... */ }
});

// Rollback design document
await helpers.rollbackDesignDoc('_design/myapp');

// Wait for view builds
await helpers.waitForViews('_design/myapp', 300000);

// Create Mango index
await helpers.createIndex({
  name: 'by-email',
  fields: ['email'],
  unique: true
});

// Drop index
await helpers.dropIndex('by-email');

// List all indexes
const indexes = await helpers.listIndexes();

// Document transformations with batching
await helpers.transformDocuments({
  selector: { type: 'user' },
  batchSize: 100,
  transform: (doc) => {
    doc.migrated = true;
    return doc;
  }
});

// Deploy validation function
await helpers.deployValidation(validationFunction);

// Update security
await helpers.updateSecurity({
  admins: { names: ['admin'], roles: [] },
  members: { names: [], roles: ['user'] }
});

// Count documents
const count = await helpers.countDocuments({ type: 'user' });

// Database operations
await helpers.createDatabase('new-db');
await helpers.deleteDatabase('old-db');
const exists = await helpers.databaseExists('my-db');

CLI Commands Reference

Project Setup

# Initialize new project
couchdb-migrate init [options]
  --typescript    Use TypeScript config (default)
  --javascript    Use JavaScript config
  --force         Overwrite existing files

Schema Commands

# Initialize schema directory
couchdb-migrate schema:init

# Create new schema
couchdb-migrate schema:create <name> [options]
  --type <type>   Document type name

# Validate all schemas
couchdb-migrate schema:validate

# Show schema changes
couchdb-migrate schema:diff

Migration Commands

# Generate migration from schema changes
couchdb-migrate generate [options]
  --dry-run       Show what would be generated
  --no-prompt     Use defaults for breaking changes
  --force         Regenerate even if no changes

# Create manual migration
couchdb-migrate create <name> [options]
  --type <type>   Migration type (default: manual)

# Show migration status
couchdb-migrate status [options]
  --json          Output as JSON
  --verbose       Show detailed information

# Run pending migrations
couchdb-migrate up [options]
  --to <id>       Migrate up to specific migration
  --dry-run       Show what would run
  --force         Skip checksum validation

# Rollback migrations
couchdb-migrate down [options]
  --to <id>       Rollback to specific migration
  --steps <n>     Number of migrations to rollback (default: 1)
  --dry-run       Show what would rollback
  --force         Skip confirmation

Global Options

--config <path>   Path to config file
--env <name>      Environment name
--verbose         Verbose logging
--quiet           Minimal output
--json            JSON output format

Real-World Examples

Example 1: Adding Email Verification

Step 1: Update schema

{
  "properties": {
    "emailVerified": {
      "type": "boolean",
      "default": false
    },
    "emailVerifiedAt": {
      "type": "string",
      "format": "date-time"
    }
  },
  "required": ["_id", "type", "email", "emailVerified"]
}

Step 2: Generate migration

couchdb-migrate generate

Step 3: Customize migration

async up(context: MigrationContext): Promise<void> {
  const { helpers, logger } = context;

  logger.info('Adding email verification fields');

  await helpers.transformDocuments({
    selector: { type: 'user' },
    batchSize: 100,
    transform: (doc) => {
      doc.emailVerified = false;
      doc.emailVerifiedAt = null;
      return doc;
    }
  });
}

Step 4: Run migration

couchdb-migrate up

Example 2: Migrating to New Index Structure

async up(context: MigrationContext): Promise<void> {
  const { helpers, logger } = context;

  // Create new compound index
  await helpers.createIndex({
    name: 'by-status-date',
    fields: ['status', 'createdAt']
  });

  logger.info('Waiting for index to build...');

  // Drop old single-field index
  await helpers.dropIndex('by-status');

  logger.info('Index migration complete');
}

Example 3: Design Document Update

async up(context: MigrationContext): Promise<void> {
  const { helpers, logger } = context;

  logger.info('Deploying new views');

  const result = await helpers.deployDesignDoc({
    _id: '_design/analytics',
    views: {
      usersByDate: {
        map: `function(doc) {
          if (doc.type === 'user') {
            emit(doc.createdAt, {
              email: doc.email,
              name: doc.name
            });
          }
        }`,
        reduce: '_count'
      },
      activeUsers: {
        map: `function(doc) {
          if (doc.type === 'user' && doc.active) {
            emit(doc.email, null);
          }
        }`
      }
    }
  });

  logger.info(`Views built in ${result.viewBuildTimeMs}ms`);
}

Production Deployment

Environment Configuration

// couchdb-migrate.config.ts
export default {
  environments: {
    development: {
      connection: {
        url: 'http://localhost:5984',
        database: 'myapp_dev'
      }
    },
    staging: {
      connection: {
        url: process.env.STAGING_COUCHDB_URL,
        database: 'myapp_staging'
      }
    },
    production: {
      connection: {
        url: process.env.PROD_COUCHDB_URL,
        database: 'myapp_prod'
      },
      migrations: {
        validateChecksums: true,
        rollbackOnFailure: true
      }
    }
  }
}

CI/CD Integration

# .github/workflows/migrations.yml
name: Database Migrations

on:
  push:
    branches: [main]

jobs:
  migrate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: '18'

      - name: Install dependencies
        run: npm install

      - name: Run migrations (dry-run)
        run: npx couchdb-migrate up --dry-run --env production
        env:
          PROD_COUCHDB_URL: ${{ secrets.COUCHDB_URL }}

      - name: Run migrations
        run: npx couchdb-migrate up --env production
        env:
          PROD_COUCHDB_URL: ${{ secrets.COUCHDB_URL }}

Backup Strategy

# Before running migrations in production
couchdb-migrate up --backup

# This creates:
# - backup-20260125120000.tar.gz
# - Stored in ./backups/
# - Auto-cleanup after retention period

Monitoring

# Check status
couchdb-migrate status --json

# Output:
{
  "applied": [
    {
      "id": "20260125120000-schema-changes",
      "appliedAt": "2026-01-25T12:00:00.000Z",
      "executionTimeMs": 45
    }
  ],
  "pending": [],
  "total": 1
}

Best Practices

Schema Design

1. Use descriptive field names and add descriptions
2. Define validation constraints (minLength, maxLength, pattern)
3. Version your schemas using x-couchdb.version
4. Document breaking changes in schema comments
5. Use consistent naming (camelCase for fields, kebab-case for IDs)

Migration Management

1. Never modify applied migrations - Create new ones instead
2. Test migrations locally before production
3. Always provide down() functions for rollback
4. Use small, focused migrations rather than large ones
5. Document manual steps in migration comments
6. Test rollback before deploying

Version Control

# Commit pattern
git add schemas/ migrations/
git commit -m "feat: add email verification to users"

# Don't commit
.gitignore:
generated/
.couchdb-migrate/
docs/

Testing

# Test migration locally
couchdb-migrate up --config test.config.ts

# Run dry-run first
couchdb-migrate up --dry-run

# Verify with status
couchdb-migrate status --verbose

Performance

1. Use appropriate batch sizes (default: 100 documents)
2. Monitor view build times and schedule during off-peak
3. Use indexes wisely - Too many can slow writes
4. Consider document size when transforming
5. Test with production-sized datasets

Troubleshooting

Common Issues

Migration fails with "Lock timeout"

# Someone else is running migrations
# Wait for completion or force release (dangerous!)
couchdb-migrate status

"Checksum mismatch" error

# Migration file was modified after being applied
# Options:
# 1. Revert the file to original state
# 2. Use --force (not recommended)
# 3. Create a new migration for the changes

Views taking too long to build

// Increase timeout in migration
await helpers.deployDesignDoc(ddoc, {
  viewTimeout: 600000  // 10 minutes
});

"Schema not found" error

# Verify schema file exists and is valid JSON
couchdb-migrate schema:validate

Debug Mode

# Enable verbose logging
couchdb-migrate up --verbose

# Or set environment variable
DEBUG=couchdb-migrate:* couchdb-migrate up

Getting Help

# Command help
couchdb-migrate --help
couchdb-migrate up --help

# Version
couchdb-migrate --version

API Reference

Configuration

interface MigrateConfig {
  connection: {
    url: string;
    database: string;
    username?: string;
    password?: string;
  };

  paths: {
    schemas: string;
    migrations: string;
    generated: string;
  };

  migrations?: {
    stateDocumentId?: string;
    lockDocumentId?: string;
    lockTTL?: number;
    lockHeartbeatInterval?: number;
    rollbackOnFailure?: boolean;
    validateChecksums?: boolean;
  };

  execution?: {
    batchSize?: number;
    viewBuildTimeout?: number;
    retryAttempts?: number;
    retryDelay?: number;
  };
}

Migration Interface

interface Migration {
  id: string;
  name: string;
  type: MigrationType;
  up: (context: MigrationContext) => Promise<void>;
  down?: (context: MigrationContext) => Promise<void>;
}

interface MigrationContext {
  db: CouchDBClient;
  helpers: MigrationHelpers;
  logger: Logger;
  config: MigrateConfig;
}

Schema Interface

interface JSONSchema {
  $schema: string;
  $id: string;
  title?: string;
  description?: string;
  type: string;
  properties?: Record<string, JSONSchema>;
  required?: string[];
  [key: string]: any;
  'x-couchdb'?: CouchDBExtensions;
}

interface CouchDBExtensions {
  type: string;
  version: number;
  indexes?: IndexDefinition[];
  views?: ViewDefinition[];
  validation?: ValidationRules;
  security?: SecurityObject;
}

Contributing

We welcome contributions!

Development Setup

# Clone repository
git clone https://github.com/ronitkishore/couchdb-migrate.git
cd couchdb-migrate

# Install dependencies
npm install

# Run tests
npm test

# Build
npm run build

# Link for local development
npm link

Running Tests

# Unit tests
npm test

# Watch mode
npm run test:watch

# Coverage
npm run test:coverage

Code Style

• TypeScript strict mode
• ESLint + Prettier
• 100% test coverage for new features
• JSDoc comments for public APIs

License

MIT

Changelog

See CHANGELOG.md for release history.

Credits

Built with:

• nano - CouchDB client
• ajv - JSON Schema validation
• commander - CLI framework
• chalk - Terminal styling

Ready to get started?

npm install -g couchdb-migrate
couchdb-migrate init

Happy migrating!