@bernierllc/csv-import-suite

Node.js only — backend CSV import orchestration with a 6-step session state machine.

Warning: Node.js Only

This package is Node.js-only and cannot be bundled for browser use ("browser": false). It depends on server-side packages (csv-import-service, merge-planner, entity-resolver) that use Node.js built-ins.

For browser/frontend applications, use @bernierllc/csv-import-suite-client instead. That package communicates with this suite over HTTP.

Overview

@bernierllc/csv-import-suite is the backend half of a CSV import system. It manages stateful import sessions through a fixed 6-step workflow, validates data against schemas, resolves entities, builds merge plans, and commits records via a user-supplied Persister.

The suite does not start an HTTP server. You mount its handle* methods on your own Express/Fastify/Koa routes using the path constants from the contract module.

Architecture

Browser --HTTP--> Your Server --> CSVImportSuite.handle*()
                                        |
                        +---------------+---------------+
                        v               v               v
                   CsvParser      EntityResolver    MergePlanner
                   CsvMapper      CsvImportService  CsvValidator

6-Step Session State Machine

Each import session passes through these steps in order:

| Step | Method | HTTP | Description | |------|--------|------|-------------| | 1. Upload | handleCreateSession | POST /csv-imports | Parse CSV, auto-suggest column mapping | | 2. Map | handleSetMapping | PUT /csv-imports/:id/mapping | Confirm column-to-field mapping | | 3. Validate | handleValidate | POST /csv-imports/:id/validate | Run field-level validation rules | | 4. Resolve | handleResolve | POST /csv-imports/:id/resolve | Match rows against existing records | | 5. Plan | handleBuildPlan | POST /csv-imports/:id/plan | Build create/merge/skip plan from operator decisions | | 6. Commit | handleCommit | POST /csv-imports/:id/commit | Apply plan via Persister, track progress |

Steps must be called in order. Skipping or repeating steps throws INVALID_STEP.

Installation

npm install @bernierllc/csv-import-suite

Requires Node.js 18+.

Usage

import { CSVImportSuite } from '@bernierllc/csv-import-suite';
import type { CandidateProvider, UploaderSchema, Persister } from '@bernierllc/csv-import-suite';

// 1. Define your schema
const uploaderSchema: UploaderSchema = {
  fields: [
    { targetField: 'email', label: 'Email', identity: true, mergeable: false },
    { targetField: 'name', label: 'Name', identity: false, mergeable: true },
  ],
};

// 2. Supply a candidate provider (queries your DB for potential matches)
const candidateProvider: CandidateProvider = {
  async findCandidates(rows) {
    return rows.map(row => myDb.findByEmail(row['email'] as string));
  }
};

// 3. Supply a persister (writes creates/merges to your DB)
const persister: Persister = {
  async create(rows) { return myDb.insertMany(rows); },
  async merge(target, updates) { return [await myDb.update((target as { id: string }).id, updates)]; },
};

// 4. Instantiate and initialize (NeverHub auto-detected)
const suite = new CSVImportSuite();
await suite.initialize();

// 5. Mount on your HTTP router (Express example)
import express from 'express';
const app = express();
app.use(express.json({ limit: '10mb' }));

app.post('/csv-imports', async (req, res) => {
  const result = await suite.handleCreateSession(req.body, {
    uploaderSchema,
    candidateProvider,
    persister,
    matchConfig: {
      rules: [{ field: 'email', comparator: 'compareEmail', weight: 1 }],
      autoAcceptAt: 0.9,
      reviewAt: 0.6,
    },
  });
  res.json(result);
});

app.put('/csv-imports/:sessionId/mapping', async (req, res) => {
  const result = await suite.handleSetMapping(req.params['sessionId']!, req.body);
  res.json(result);
});

app.post('/csv-imports/:sessionId/validate', async (req, res) => {
  const result = await suite.handleValidate(req.params['sessionId']!, req.body);
  res.json(result);
});

app.post('/csv-imports/:sessionId/resolve', async (req, res) => {
  const result = await suite.handleResolve(req.params['sessionId']!, req.body);
  res.json(result);
});

app.post('/csv-imports/:sessionId/plan', async (req, res) => {
  const result = await suite.handleBuildPlan(req.params['sessionId']!, req.body);
  res.json(result);
});

app.post('/csv-imports/:sessionId/commit', async (req, res) => {
  const result = await suite.handleCommit(req.params['sessionId']!, req.body);
  res.json(result);
});

app.get('/csv-imports/:sessionId/progress', (req, res) => {
  const result = suite.handleProgress(req.params['sessionId']!);
  res.json(result);
});

HTTP Contract

Import path constants and zod schemas from the contract module to validate requests/responses independently:

import { PATHS, CONTRACT_SCHEMAS, CreateSessionRequestSchema } from '@bernierllc/csv-import-suite';

// Path helpers
PATHS.createSession              // '/csv-imports'
PATHS.setMapping('sess-123')     // '/csv-imports/sess-123/mapping'
PATHS.progress('sess-123')       // '/csv-imports/sess-123/progress'

// Schema validation
const parsed = CreateSessionRequestSchema.safeParse(body);

Contract Operations

| Operation | Request Schema | Response Schema | |-----------|---------------|----------------| | createSession | CreateSessionRequestSchema | CreateSessionResponseSchema | | setMapping | SetMappingRequestSchema | SetMappingResponseSchema | | validate | ValidateRequestSchema | ValidateResponseSchema | | resolve | ResolveRequestSchema | ResolveResponseSchema | | buildPlan | BuildPlanRequestSchema | BuildPlanResponseSchema | | commit | CommitRequestSchema | CommitResponseSchema | | progress | — | ProgressResponseSchema |

API Reference

new CSVImportSuite<Row, Existing>(config?)

Creates a new suite instance. All config fields are optional — pass collaborators at session-creation time instead.

const suite = new CSVImportSuite({
  defaults: {
    uploaderSchema,   // Applied to all sessions unless overridden per-session
    matchConfig,
    candidateProvider,
    persister,
  }
});

suite.initialize(): Promise<void>

Auto-detects NeverHub and registers. Safe to call multiple times (idempotent). Offline-safe — the suite works without NeverHub.

suite.handleCreateSession(body, overrides?): Promise<CreateSessionResponse>

Step 1. Parses CSV, suggests column mapping. Returns sessionId, headers, and suggestions.

suite.handleSetMapping(sessionId, body): Promise<SetMappingResponse>

Step 2. Stores the operator-confirmed mapping (CSV column to target field).

suite.handleValidate(sessionId, body): Promise<ValidateResponse>

Step 3. Validates each row against the uploader schema's field rules. Returns per-row results with errors.

suite.handleResolve(sessionId, body): Promise<ResolveResponse>

Step 4. Runs entity resolution. Returns per-row resolution results with tier (accept, review, or none), best match, conflicts, and alternatives.

suite.handleBuildPlan(sessionId, body): Promise<BuildPlanResponse>

Step 5. Accepts operator decisions (create/merge/skip per row, with field-level conflict resolution) and builds a MergePlan. Returns summary counts.

suite.handleCommit(sessionId, body): Promise<CommitResponse>

Step 6. Applies the plan via Persister. Tracks progress in real time. Returns final counts (applied, skipped, failed, total).

suite.handleProgress(sessionId): ProgressResponse

Returns real-time progress snapshot for a running commit. Safe to poll.

suite.getSession(sessionId): ImportSession | undefined

Returns the current session state (in-memory). Useful for debugging.

suite.isNeverHubAvailable(): boolean

Returns whether NeverHub was detected and registered successfully.

Error Handling

All errors are instances of CsvImportSuiteError with a code property:

import { CsvImportSuiteError } from '@bernierllc/csv-import-suite';

try {
  await suite.handleCommit(sessionId, {});
} catch (err) {
  if (err instanceof CsvImportSuiteError) {
    switch (err.code) {
      case 'SESSION_NOT_FOUND':           // sessionId not recognized
      case 'INVALID_STEP':                // wrong step order
      case 'INVALID_INPUT':               // missing required collaborator
      case 'CONTRACT_VALIDATION_FAILED':  // request body failed zod schema
      case 'PARSE_FAILED':                // CSV could not be parsed
      case 'MAPPING_FAILED':              // column mapping invalid
      case 'VALIDATION_FAILED':           // field validation error
      case 'RESOLUTION_FAILED':           // entity resolution threw
      case 'PLAN_BUILD_FAILED':           // merge plan construction failed
      case 'COMMIT_FAILED':               // persister or applyMergePlan threw
      case 'SCHEMA_NOT_FOUND':            // import schema not in storage
      case 'STORAGE_ERROR':               // serialization/deserialization error
      case 'NEVERHUB_ERROR':              // NeverHub registration failed
    }
  }
}

Schema Management

The suite includes a SchemaManager for CRUD operations on ImportSchema objects stored in ImportStorage:

import { SchemaManager, ImportStorage } from '@bernierllc/csv-import-suite';

const storage = new ImportStorage({ provider: 'memory' });
const manager = new SchemaManager(storage);

const { schemaId } = await manager.createSchema({
  name: 'Contact Import',
  fields: [
    { name: 'email', type: 'email', required: true },
    { name: 'name', type: 'string', required: true },
    { name: 'phone', type: 'phone' },
  ],
});

NeverHub Integration

The suite registers with NeverHub automatically if available. This is non-blocking — the import workflow works identically whether NeverHub is online or not.

const suite = new CSVImportSuite();
await suite.initialize();

if (suite.isNeverHubAvailable()) {
  console.log('Registered with NeverHub');
}

React Migration Guide

Prior to v1.0.0, this package exported React components (ImportWizard, ImportManager, SchemaEditor). These components have been removed in v1.0.0.

Migration path:

1. Install @bernierllc/csv-import-suite-client for browser/React applications
2. Mount this package's HTTP handlers on your backend
3. Use the client package to call those handlers from your React app

License

Copyright (c) 2025 Bernier LLC. Licensed under a limited-use license — see LICENSE for details.