@sops-node/migration-engine

A production-ready TypeScript/Node.js library for comparing and migrating complex object schemas with path-based diffing. It features a layered Approval System to ensure safe and intentional data transformations.

Features

• Deep Recursive Diffing: Path-based comparison (e.g., user.profile.meta.verified).
• Recursive Array Diffing: Index-based comparison for arrays (e.g., list[0]).
• Layered Approval System:
  • Global Plan Approval: Master switch for the entire migration.
  • Per-Change Approval: Granular control over each ADD, UPDATE, and DELETE.
• Conflict Detection: Identifies type mismatches and structural inconsistencies (e.g., Object vs Primitive).
• Safe Execution: Immutable updates powered by lodash deep clones.
• Bulk Utilities: Simple API to approve or reject multiple changes at once.
• CLI Tool: Pretty console output with colored status logs.
• Dual ESM/CJS Support: Bundled with tsup for universal compatibility.

Installation

npm install @sops-node/migration-engine

Basic Usage

import { calculateDiff, createMigrationPlan, executeMigration, applyApprovalStatus } from '@sops-node/migration-engine';

const oldObj = { profile: { name: "Alice" } };
const newObj = { profile: { name: "Bob", age: 30 } };

// 1. Calculate the raw difference
// Returns DiffChange[] (no approval status yet)
const changes = calculateDiff(oldObj, newObj);

// 2. Wrap changes with approval capability (optional)
// Returns ApprovedDiffChange[] (initialized to false)
const preparedChanges = applyApprovalStatus(changes, false);

// 3. Create a migration plan (groups and initializes approval capability)
// Returns MigrationPlan
const plan = createMigrationPlan(changes, true); // true sets plan.approved = true

// 4. Safely execute the migration
const result = executeMigration(oldObj, plan); 

console.log(result);
// { profile: { name: "Bob", age: 30 } }

Approval Workflow

The engine enforces a Safety-First approach. By default, all detected changes in a MigrationPlan are unapproved (approvedStatus: false).

Granular Approval

You can selectively approve individual changes after reviewing them:

const changes = calculateDiff(old, new);
const plan = createMigrationPlan(changes, true); 

// Approve only specific paths
plan.updates.forEach(change => {
  if (change.path === 'user.email') {
    change.approvedStatus = true;
  }
});

const result = executeMigration(old, plan); 
// Only user.email was updated; other changes were skipped!

Safety Rules

1. If plan.approved is false, executeMigration will throw an Error.
2. Even if plan.approved is true, individual changes will only be applied if their change.approvedStatus flag is true.

Array Support

The engine natively diffs arrays using index-based tracking.

• Path Example: users[0].name
• Detection: It distinguishes between an update to an existing element, an addition at the end, or a deletion.

API Reference

calculateDiff(oldSchema, newSchema)

Returns a flat array of DiffChange objects.

applyApprovalStatus(changes, approved)

Converts DiffChange[] to ApprovedDiffChange[] and sets the approvedStatus.

createMigrationPlan(changes, approved = false)

Groups raw changes into adds, updates, deletes, and conflicts, promoting them to ApprovedDiffChange.

executeMigration(oldObject, plan)

Applies approved changes to a clone of the oldObject. Throws if the plan approved flag is false.

Types

DiffChange

interface DiffChange {
  type: 'ADD' | 'UPDATE' | 'DELETE' | 'CONFLICT';
  path: string;
  oldValue?: any;
  newValue?: any;
  message?: string;
}

ApprovedDiffChange

interface ApprovedDiffChange extends DiffChange {
  approvedStatus: boolean; // Must be true for execution
}

MigrationPlan

interface MigrationPlan {
  adds: ApprovedDiffChange[];
  updates: ApprovedDiffChange[];
  deletes: ApprovedDiffChange[];
  conflicts: ApprovedDiffChange[];
  approved: boolean; // Global master switch
}

CLI Usage

Diffing

npx migration-engine diff old.json new.json

Applying

npx migration-engine apply old.json new.json --output migrated.json

License

MIT