@arraypress/csv-importer

Parse → preview → commit scaffold for CSV-driven entity imports. Bundles the standard 3-stage pipeline shape across transactions / customers / products / files / whatever-you-have importers, with auto-stats computation and standard decision builders.

Lightweight (~100 LOC of orchestration). The library handles the glue; you write the per-type preview + commit logic.

What this gives you

• Type-dispatcher — register { types: { customers, products, files, ... } }, each with a preview + commit function. The library routes calls to the right one.
• Standard pipeline shape — parse({ csv }) → preview({ type, csv, mapping }) → commit({ type, csv, mapping }). Same shape across every importer in your app.
• Auto-tallied stats — preview returns { decisions, stats } where stats is { create: 12, update: 3, skip: 1 } aggregated from the per-row decisions. No manual histogram code.
• Decision builders — decisionCreate, decisionUpdate, decisionSkip, decisionDuplicate, decisionMatch to keep the per-row return value uniform.

Install

npm install @arraypress/csv-importer

Quick start

import {
  createImporter,
  decisionCreate, decisionUpdate, decisionSkip, decisionDuplicate,
} from '@arraypress/csv-importer';

const importer = createImporter({
  types: {
    customers: {
      async preview({ rows, mapping, db }) {
        const existing = new Set(
          (await db.selectFrom('customers').select('email').execute()).map((r) => r.email),
        );
        const seen = new Set();
        return rows.map((row, i) => {
          const email = (row[mapping.email] || '').toLowerCase().trim();
          if (!email.includes('@')) return decisionSkip(i, 'invalid email', { email });
          if (seen.has(email)) return decisionDuplicate(i, 'duplicate in CSV', { email });
          seen.add(email);
          return existing.has(email)
            ? decisionUpdate(i, { email })
            : decisionCreate(i, { email });
        });
      },
      async commit({ rows, mapping, db }) {
        // ... persist ...
        return { total: rows.length, created: 5, updated: 2, skipped: 1, skippedRows: [] };
      },
    },
    // products, files, transactions ... same shape
  },
});

// In your route handlers:
app.post('/imports/parse', async (c) => {
  const { csv, headerRowIndex } = await c.req.json();
  return c.json(importer.parse({ csv, headerRowIndex }));
});

app.post('/imports/preview', async (c) => {
  const { type, csv, headerRowIndex, mapping } = await c.req.json();
  if (!importer.isKnownType(type)) return c.json({ error: 'unknown type' }, 400);
  const result = await importer.preview({ type, csv, headerRowIndex, mapping, db: getDb(c.env.DB) });
  return c.json(result);
});

app.post('/imports/commit', async (c) => {
  const { type, csv, headerRowIndex, mapping, source } = await c.req.json();
  if (!importer.isKnownType(type)) return c.json({ error: 'unknown type' }, 400);
  const result = await importer.commit({ type, csv, headerRowIndex, mapping, source, db: getDb(c.env.DB) });
  return c.json(result);
});

API

createImporter({ types })

Registers a map of type → { preview, commit } and returns a dispatcher with parse, preview, commit, isKnownType, and knownTypes methods.

parse({ csv, headerRowIndex? })

Wraps @arraypress/csv/parse — auto-detects the header row when headerRowIndex isn't supplied, returns headers + sample rows + total count for the column-mapping UI.

preview({ type, csv, headerRowIndex?, mapping, db })

Calls the per-type preview function, then computes the action histogram. Returns { type, totalRows, decisions, stats }.

commit({ type, csv, headerRowIndex?, mapping, db, source? })

Calls the per-type commit function and returns its result verbatim. The library doesn't impose a result shape — ImportResult is recommended but per-type extensions are fine (e.g. transactions importers often add createdCustomers, createdProducts).

tally(items, key?)

Helper for any histogram — counts items by their key field (default action). Used internally by preview; exported because consumers occasionally want the same histogram outside the importer.

Decision builders

| Builder | Action | Use case | |---|---|---| | decisionCreate(rowIndex, extra?) | 'create' | Row will create a new record. | | decisionUpdate(rowIndex, extra?) | 'update' | Row will update an existing record. | | decisionSkip(rowIndex, reason, extra?) | 'skip' | Row was skipped — reason shown in the per-row error list. | | decisionDuplicate(rowIndex, reason, extra?) | 'duplicate' | Row is a duplicate of an earlier row in the same CSV. | | decisionMatch(rowIndex, via, extra?) | 'match' | Row matched an existing record via some fuzzy/multi-tier key (e.g. SugarVault's product matcher). |

The extra object spreads onto the result so you can attach display data:

decisionCreate(0, { email: '[email protected]' });
// → { rowIndex: 0, action: 'create', email: '[email protected]' }

Per-type Importer interface

interface Importer<Db, Result> {
  preview(args: {
    rows: Record<string, string>[];
    mapping: Record<string, string>;
    db: Db;
  }): Promise<Decision[]> | Decision[];

  commit(args: {
    rows: Record<string, string>[];
    mapping: Record<string, string>;
    db: Db;
    source?: string;
  }): Promise<Result>;
}

Patterns

Recommended commit return shape

ImportResult-shaped objects make UI parity easier:

interface ImportResult {
  total: number;
  created: number;
  updated: number;
  skipped: number;
  skippedRows: Array<{ row: number; reason: string }>;
}

Add per-type fields on top — SugarVault's transactions importer extends with createdCustomers, createdProducts, createdTransactions, skippedTransactions.

Pre-loading for hot per-type previews

The preview callback runs once per request — cache anything DB-heavy inside it (the existing-emails set, the slug list, etc.) to avoid an N+1 over rows:

async preview({ rows, mapping, db }) {
  const existing = new Set(
    (await db.selectFrom('customers').select('email').execute()).map((r) => r.email),
  );
  return rows.map((row, i) => /* … cheap in-memory check using existing … */);
}

Validation

The dispatcher throws when type isn't in the registered set. Use isKnownType() upstream in your route handler to return 400 before hitting preview / commit:

if (!importer.isKnownType(type)) return c.json({ error: 'unknown type' }, 400);

License

MIT