@migrex/core

Core library for versioned data contracts with bidirectional migrations.

Overview

Migrex provides a type-safe, graph-based migration system for evolving data schemas over time. It enables you to:

• Define multiple versions of your data schemas
• Write bidirectional migrations between versions
• Automatically compute migration paths between any two versions
• Preserve fields during migrations that would otherwise be lost
• Analyze your migration graph for connectivity and data loss issues

Key Concepts

Versioned Data Contracts: Each version of your data has a schema that validates and types the data. Versions are ordered using a strategy (semver, calver, or integer).

Bidirectional Migrations: Most migrations are reversible, allowing you to migrate both forward (up) and backward (down) between versions. This ensures data can flow in both directions without loss.

One-Way Migrations: Some transformations are intentionally lossy or irreversible. These are marked with traits to help users understand the implications.

Field Preservation: When a field is removed in one version but needed in a later version, migrations can preserve the field in a "stash" and restore it later. This enables lossless multi-step migrations.

Migration Graph: All versions and migrations form a directed graph. Migrex automatically finds the shortest path between any two versions.

Installation

pnpm add @migrex/core

Quick Start

import { createMigrationGraph, semverStrategy } from '@migrex/core';
import type { VersionedSchema, ReversibleMigration, MigrationContext } from '@migrex/core';

// Define your data types
type UserV1 = {
  version: '1.0.0';
  name: string;
};

type UserV2 = {
  version: '2.0.0';
  firstName: string;
  lastName: string;
};

// Define schemas with validation
const schemaV1: VersionedSchema<UserV1> = {
  version: '1.0.0',
  schema: (data: unknown) => {
    // Validate the shape and return typed data
    if (
      typeof data === 'object' &&
      data !== null &&
      'name' in data &&
      typeof data.name === 'string' &&
      'version' in data &&
      data.version === '1.0.0'
    ) {
      return { success: true, data: data as UserV1 };
    }
    return {
      success: false,
      errors: [{ path: [], message: 'Invalid v1 user data' }],
    };
  },
};

const schemaV2: VersionedSchema<UserV2> = {
  version: '2.0.0',
  schema: (data: unknown) => {
    if (
      typeof data === 'object' &&
      data !== null &&
      'firstName' in data &&
      typeof data.firstName === 'string' &&
      'lastName' in data &&
      typeof data.lastName === 'string' &&
      'version' in data &&
      data.version === '2.0.0'
    ) {
      return { success: true, data: data as UserV2 };
    }
    return {
      success: false,
      errors: [{ path: [], message: 'Invalid v2 user data' }],
    };
  },
};

// Define a bidirectional migration
const migrationV1ToV2: ReversibleMigration<UserV1, UserV2> = {
  fromVersion: '1.0.0',
  toVersion: '2.0.0',
  up: (data: UserV1, _ctx: MigrationContext) => {
    // Split name into firstName and lastName
    const parts = data.name.split(' ');
    return {
      version: '2.0.0',
      firstName: parts[0] || '',
      lastName: parts.slice(1).join(' ') || '',
    };
  },
  down: (data: UserV2, _ctx: MigrationContext) => {
    // Combine firstName and lastName back into name
    return {
      version: '1.0.0',
      name: `${data.firstName} ${data.lastName}`.trim(),
    };
  },
};

// Create the migration graph
const graph = createMigrationGraph({
  id: 'user-graph',
  versionStrategy: semverStrategy,
});

// Register schemas and migrations
graph.registerSchema(schemaV1);
graph.registerSchema(schemaV2);
graph.registerMigration(migrationV1ToV2);

// Migrate data from v1 to v2
const v1Data: UserV1 = { version: '1.0.0', name: 'John Doe' };
const result = graph.migrate(v1Data, '1.0.0', '2.0.0');

if (result.success) {
  console.log(result.data);
  // { version: '2.0.0', firstName: 'John', lastName: 'Doe' }
}

// Migrate back from v2 to v1
const v2Data: UserV2 = { version: '2.0.0', firstName: 'Jane', lastName: 'Smith' };
const reverseResult = graph.migrate(v2Data, '2.0.0', '1.0.0');

if (reverseResult.success) {
  console.log(reverseResult.data);
  // { version: '1.0.0', name: 'Jane Smith' }
}

Declarative Graph Configuration

You can register schemas and migrations during graph creation for a more declarative style:

import { createMigrationGraph, semverStrategy } from '@migrex/core';

const graph = createMigrationGraph({
  id: 'user-graph',
  versionStrategy: semverStrategy,
  schemas: [schemaV1, schemaV2, schemaV3],
  migrations: [migrationV1ToV2, migrationV2ToV3],
});

// No need to call registerSchema() or registerMigration()!

This is equivalent to calling registerSchema() and registerMigration() for each item. You can also mix declarative and imperative registration:

const graph = createMigrationGraph({
  id: 'user-graph',
  versionStrategy: semverStrategy,
  schemas: [schemaV1, schemaV2],
});

// Add more later
graph.registerSchema(schemaV3);
graph.registerMigration(migrationV2ToV3);

Migration Helpers

For common migration patterns, use the built-in helper functions instead of writing custom migrations:

addField

Add a field with a default value:

import { addField } from '@migrex/core';

interface V1 { version: string; name: string; }
interface V2 { version: string; name: string; logLevel: string; }

const migration = addField<V1, V2, 'logLevel'>({
  from: '1.0.0',
  to: '1.1.0',
  field: 'logLevel',
  defaultValue: 'info',
});

graph.registerMigration(migration);

removeField

Remove a field, optionally preserving it for later restoration:

import { removeField } from '@migrex/core';

interface V1 { version: string; name: string; deprecated: string; }
interface V2 { version: string; name: string; }

// Simple removal (lossy)
const migration = removeField<V1, V2, 'deprecated'>({
  from: '1.0.0',
  to: '1.1.0',
  field: 'deprecated',
});

// With preservation (lossless)
const preservingMigration = removeField<V1, V2, 'deprecated'>({
  from: '1.0.0',
  to: '1.1.0',
  field: 'deprecated',
  preserveAs: 'old.deprecated', // Stored in the stash
});

renameField

Rename a field while preserving its value:

import { renameField } from '@migrex/core';

interface V1 { version: string; user_name: string; }
interface V2 { version: string; userName: string; }

const migration = renameField<V1, V2>({
  from: '1.0.0',
  to: '2.0.0',
  oldName: 'user_name',
  newName: 'userName',
});

restructure

Transform between flat and nested structures:

import { restructure } from '@migrex/core';

interface V1 { version: string; appName: string; port: number; }
interface V2 { version: string; app: { name: string }; server: { port: number }; }

const migration = restructure<V1, V2>({
  from: '1.0.0',
  to: '2.0.0',
  transform: {
    'app.name': 'appName',
    'server.port': 'port',
  },
});

The transform object maps nested paths (in v2) to flat field names (in v1). This helper:

• Automatically handles both up (flat → nested) and down (nested → flat) migrations
• Sets appropriate traits to indicate the structural transformation
• Is fully reversible

Field Preservation

When a field is removed in one version but needed later, you can preserve it:

type V1 = { version: '1.0.0'; name: string; theme: 'light' | 'dark' };
type V2 = { version: '2.0.0'; name: string }; // theme removed
type V3 = { version: '3.0.0'; name: string; theme: 'light' | 'dark' }; // theme restored

const migrationV1ToV2: ReversibleMigration<V1, V2> = {
  fromVersion: '1.0.0',
  toVersion: '2.0.0',
  up: (data: V1, ctx: MigrationContext) => {
    // Preserve theme for later restoration
    ctx.preserve('theme', data.theme, {
      sourceVersion: '1.0.0',
      description: 'Theme preference reintroduced in v3',
    });

    return {
      version: '2.0.0',
      name: data.name,
    };
  },
  down: (data: V2, ctx: MigrationContext) => {
    // Restore theme when migrating back
    const preserved = ctx.consume('theme');
    return {
      version: '1.0.0',
      name: data.name,
      theme: (preserved?.value as 'light' | 'dark') ?? 'light',
    };
  },
};

const migrationV2ToV3: ReversibleMigration<V2, V3> = {
  fromVersion: '2.0.0',
  toVersion: '3.0.0',
  up: (data: V2, ctx: MigrationContext) => {
    // Restore preserved theme
    const preserved = ctx.consume('theme');
    return {
      version: '3.0.0',
      name: data.name,
      theme: (preserved?.value as 'light' | 'dark') ?? 'light',
    };
  },
  down: (data: V3, ctx: MigrationContext) => {
    // Preserve theme when migrating back through v2
    ctx.preserve('theme', data.theme, {
      sourceVersion: '3.0.0',
      description: 'Theme exists in v1',
    });

    return {
      version: '2.0.0',
      name: data.name,
    };
  },
};

// Now v1 -> v2 -> v3 migration preserves the theme field
const graph = createMigrationGraph({
  id: 'theme-example',
  versionStrategy: semverStrategy,
});
graph.registerSchema(schemaV1);
graph.registerSchema(schemaV2);
graph.registerSchema(schemaV3);
graph.registerMigration(migrationV1ToV2);
graph.registerMigration(migrationV2ToV3);

const v1Data: V1 = { version: '1.0.0', name: 'Alice', theme: 'dark' };
const result = graph.migrate(v1Data, '1.0.0', '3.0.0');

if (result.success) {
  console.log(result.data);
  // { version: '3.0.0', name: 'Alice', theme: 'dark' }
  // Theme was preserved through v2!

  console.log(result.stash.consumed);
  // ['theme'] - shows theme was successfully consumed in v3

  console.log(result.stash.lossless);
  // true - no data was lost during migration
}

Round-Trip Migrations

When chaining multiple migrations (e.g., forward then backward), pass the stash from the first migration to the second using initialStash:

// Migrate v1 -> v3 (forward)
const v1Data: V1 = { version: '1.0.0', name: 'Alice', theme: 'dark' };
const forwardResult = graph.migrate(v1Data, '1.0.0', '3.0.0');

if (forwardResult.success) {
  // Now migrate v3 -> v1 (backward), passing the stash
  const backwardResult = graph.migrate(forwardResult.data, '3.0.0', '1.0.0', {
    initialStash: forwardResult.preservedFields,
  });

  if (backwardResult.success) {
    console.log(backwardResult.data);
    // { version: '1.0.0', name: 'Alice', theme: 'dark' }
    // Theme was preserved through the round-trip!
  }
}

Graph Analysis

Analyze your migration graph to find connectivity issues, lossy paths, and optimization opportunities:

import { analyzeGraph, findLossyPaths, suggestMigrations } from '@migrex/core';

// Analyze the graph
const analysis = analyzeGraph(graph);

console.log(`Analyzed ${analysis.metadata.versionCount} versions`);
console.log(`Found ${analysis.diagnostics.length} issues`);

// Check connectivity
if (!analysis.connectivity.fullyConnected) {
  console.log('Warning: Graph has disconnected components');
  for (const component of analysis.connectivity.components) {
    console.log(`  Component: ${component.join(', ')}`);
  }
}

// Check for lossy paths
const lossyReport = findLossyPaths(graph);
console.log(`\nLossy paths: ${lossyReport.lossyPairCount}/${lossyReport.totalVersionPairs}`);

for (const { from, to, analysis } of lossyReport.lossyPairs) {
  console.log(`\n${from} -> ${to}: Data loss detected`);
  for (const step of analysis.lossySteps) {
    console.log(`  ${step.from} -> ${step.to}: ${step.reason}`);
  }
}

// Get migration suggestions
const suggestions = suggestMigrations(graph);
for (const suggestion of suggestions) {
  console.log(`\nSuggestion: Add migration ${suggestion.from} -> ${suggestion.to}`);
  console.log(`Reason: ${suggestion.reason}`);
}

// Check for deprecated versions
import { getDeprecatedVersions } from '@migrex/core';

const deprecated = getDeprecatedVersions(graph);
for (const info of deprecated) {
  console.log(`\n${info.version} is deprecated`);
  console.log(`  Successor: ${info.deprecated.successor}`);
  console.log(`  Since: ${info.deprecated.since}`);
  if (info.deprecated.message) {
    console.log(`  Message: ${info.deprecated.message}`);
  }
}

Version Strategies

Migrex supports three versioning strategies:

Semantic Versioning (semver)

import { createMigrationGraph, semverStrategy } from '@migrex/core';

const graph = createMigrationGraph({
  id: 'my-graph',
  versionStrategy: semverStrategy,
});
// Versions: '1.0.0', '1.1.0', '2.0.0', etc.

Calendar Versioning (calver)

import { createMigrationGraph, calverStrategy } from '@migrex/core';

const graph = createMigrationGraph({
  id: 'my-graph',
  versionStrategy: calverStrategy,
});
// Versions: '2024.01.15', '2024.02.01', etc.

Integer Versioning

import { createMigrationGraph, integerStrategy } from '@migrex/core';

const graph = createMigrationGraph({
  id: 'my-graph',
  versionStrategy: integerStrategy,
});
// Versions: '1', '2', '3', etc.

API Reference

Core Types

MigrationResult<T>

Result returned by graph.migrate():

interface MigrationResult<T> {
  success: boolean;              // Whether migration succeeded
  data?: T;                      // Migrated data (if success)
  error?: Error;                 // Error (if !success)
  path: MigrationPath;           // The path that was executed
  stash: StashSummary;           // Summary of preserved fields
  preservedFields: PreservedField[]; // Raw stash for chaining migrations
}

MigrationPath

The path between two versions:

interface MigrationPath {
  steps: MigrationStep[];        // Individual migration steps
  direction: 'up' | 'down' | 'mixed';
  hasIrreversibleStep: boolean;
}

MigrationStep

A single step in the migration path:

interface MigrationStep {
  fromVersion: string;           // Source version
  toVersion: string;             // Target version
  direction: 'up' | 'down';
  migration: AnyMigration;
}

PreservationMetadata

Metadata for ctx.preserve():

interface PreservationMetadata {
  sourceVersion?: string;        // Version where field originated
  description?: string;          // Human-readable description
  fieldSchema?: FieldSchema;     // Optional validation schema
}

MigrateOptions

Options for graph.migrate():

interface MigrateOptions {
  skipValidation?: boolean;      // Skip final validation (default: false)
  validateIntermediate?: boolean; // Validate each step (default: false)
  initialStash?: PreservedField[]; // Pass stash from previous migration
}

StashSummary

Summary of field preservation across a migration:

interface StashSummary {
  consumed: PreservedFieldSummary[];   // Fields that were preserved and later consumed
  unconsumed: PreservedFieldSummary[]; // Fields that were preserved but never consumed
  lossless: boolean;                   // True if all preserved fields were consumed
}

interface PreservedFieldSummary {
  path: string;           // Field path that was preserved
  preservedAt: string;    // Version where field was first preserved
  consumedAt?: string;    // Version where field was consumed (if it was)
}

Other Types

• VersionedSchema<T> - Schema definition for a version
• ReversibleMigration<From, To> - Bidirectional migration
• OneWayMigration<From, To> - Irreversible migration
• MigrationContext - Context for field preservation (preserve, consume, hasPreserved)
• AnalysisResult - Graph analysis results

Core Functions

• createMigrationGraph(options) - Create a new migration graph with { id, versionStrategy }
• graph.registerSchema(schema) - Register a version schema
• graph.registerMigration(migration) - Add a migration between versions
• graph.migrate(data, from, to) - Execute migrations
• graph.migrateAsync(data, from, to) - Execute async migrations
• analyzeGraph(graph, options?) - Analyze graph health
• findLossyPaths(graph) - Find data-losing paths
• suggestMigrations(graph) - Get improvement suggestions

Troubleshooting

"No path exists between versions"

Problem: NoPathError thrown when trying to migrate.

Solution: Add migrations to connect the versions. Use analyzeGraph() to see disconnected components.

const analysis = analyzeGraph(graph);
if (!analysis.connectivity.fullyConnected) {
  console.log('Disconnected components:', analysis.connectivity.components);
  // Add migrations to bridge the gap
}

"Field preserved multiple times"

Problem: Warning about a field having sometimes-multi profile.

Solution: Review your preservation logic. A field should only be preserved once per path.

const analysis = analyzeGraph(graph);
for (const [field, profile] of Object.entries(analysis.profiles)) {
  if (profile === 'sometimes-multi') {
    console.log(`Field "${field}" is preserved multiple times on some paths`);
    // Review migrations that preserve this field
  }
}

"Validation failed"

Problem: SchemaValidationError thrown during migration.

Solution: Ensure your migration outputs match the target schema.

// Bad: Output doesn't match schema
up: (data) => ({
  // Missing required 'version' field!
  firstName: data.name,
})

// Good: Output matches schema
up: (data) => ({
  version: '2.0.0', // Include version field
  firstName: data.name,
  lastName: '',
})

"Unconsumed preserved fields"

Problem: Field preserved but never consumed, shown in result.stash.unconsumed.

Solution: Either consume the field in a later migration or remove the preservation.

const result = graph.migrate(data, '1.0.0', '3.0.0');
if (!result.stash.lossless) {
  console.log('Unconsumed fields:', result.stash.unconsumed);
  // Add consumption in target version migration
}

Performance with large graphs

Problem: Analysis is slow with many versions.

Solution: Use incremental analysis or limit analysis scope.

import { analyzeIncremental } from '@migrex/core';

const previousAnalysis = analyzeGraph(graph);
// ... make changes ...
const updatedAnalysis = analyzeIncremental(
  graph,
  previousAnalysis,
  ['src/migrations/new-migration.ts']
);

Development

# Run tests
pnpm test

# Run type tests
pnpm test:types

# Build
pnpm build

# Generate type declarations
pnpm build:types

# Check API surface
pnpm api:check

# Update API report
pnpm api:update

# Lint
pnpm lint

License

MIT