@seedts/import

Import/export functionality for SeedTS - Load seed data from external files (JSON, CSV, YAML) and export seeded data.

Installation

npm install @seedts/import
# or
pnpm add @seedts/import
# or
yarn add @seedts/import

Features

• 📥 Import from JSON, CSV, and YAML files
• 📤 Export to JSON, CSV, and YAML formats
• ✅ Schema Validation using Zod
• 🔄 Data Transformation with custom transform functions
• 🔀 Merge Strategies (replace, append, update, upsert)
• 🎯 Type-Safe with full TypeScript support
• ⚡ Async/Await based API

Quick Start

Import from JSON

import { importJSON, loadJSON } from '@seedts/import';

// Import with validation and transformation
const result = await importJSON({
  path: './data/users.json',
  schema: userSchema,
  transform: (record) => ({
    ...record,
    createdAt: new Date()
  })
});

console.log(`Imported ${result.count} records`);

Use in Seed Definitions

import { loadJSON } from '@seedts/import';

export const CategoriesSeed = ({ adapter }) => (
  <Seed name="categories" adapter={adapter}>
    <Action>
      <Entity data={loadJSON('./data/categories.json')} />
    </Action>
  </Seed>
);

Import from CSV

import { importCSV, loadCSV } from '@seedts/import';

// Import CSV with headers
const result = await importCSV({
  path: './data/products.csv',
  delimiter: ',',
  header: true,
  dynamicTyping: true
});

// Use in seed definition
<Entity data={loadCSV('./data/products.csv', { dynamicTyping: true })} />

Import from YAML

import { importYAML, loadYAML } from '@seedts/import';

const result = await importYAML({
  path: './data/config.yaml',
  schema: configSchema
});

// Use in seed definition
<Entity data={loadYAML('./data/config.yaml')} />

Import API

importJSON(options)

Import data from a JSON file.

Options:

• path (string, required) - Path to the JSON file
• schema (ZodSchema, optional) - Zod schema for validation
• transform (function, optional) - Transform function (record) => transformedRecord
• skipInvalid (boolean, optional) - Skip invalid records instead of throwing (default: false)
• encoding (string, optional) - File encoding (default: 'utf-8')

Returns: Promise<ImportResult<T>>

interface ImportResult<T> {
  data: T[];              // Successfully imported records
  count: number;          // Number of records imported
  invalidCount: number;   // Number of records that failed validation
  errors: Array<{         // Validation errors
    index: number;
    record: any;
    error: string;
  }>;
}

Example:

import { importJSON } from '@seedts/import';
import { z } from 'zod';

const userSchema = z.object({
  name: z.string(),
  email: z.string().email(),
  age: z.number().min(18)
});

const result = await importJSON({
  path: './users.json',
  schema: userSchema,
  transform: (record) => ({
    ...record,
    createdAt: new Date(),
    updatedAt: new Date()
  }),
  skipInvalid: false
});

console.log(`Imported: ${result.count}`);
console.log(`Invalid: ${result.invalidCount}`);

importCSV(options)

Import data from a CSV file.

Options: All from importJSON plus:

• delimiter (string, optional) - Delimiter character (default: ',')
• header (boolean, optional) - First row contains headers (default: true)
• quoteChar (string, optional) - Quote character (default: '"')
• skipEmptyLines (boolean, optional) - Skip empty lines (default: true)
• dynamicTyping (boolean, optional) - Auto-detect types (default: false)

Example:

import { importCSV } from '@seedts/import';

const result = await importCSV({
  path: './products.csv',
  delimiter: ',',
  header: true,
  dynamicTyping: true,
  transform: (record) => ({
    ...record,
    price: parseFloat(record.price),
    inStock: record.inStock === 'true'
  })
});

importYAML(options)

Import data from a YAML file. Same options as importJSON.

Example:

import { importYAML } from '@seedts/import';

const result = await importYAML({
  path: './config.yaml',
  schema: configSchema
});

Helper Functions: loadJSON, loadCSV, loadYAML

These functions return a function that can be used directly in seed definitions.

loadJSON<T>(path: string): () => Promise<T[]>
loadCSV<T>(path: string, options?: CSVImportOptions): () => Promise<T[]>
loadYAML<T>(path: string): () => Promise<T[]>

Example:

import { loadJSON, loadCSV, loadYAML } from '@seedts/import';

<Entity data={loadJSON('./data/categories.json')} />
<Entity data={loadCSV('./data/products.csv', { dynamicTyping: true })} />
<Entity data={loadYAML('./data/config.yaml')} />

Export API

exportJSON(data, options)

Export data to a JSON file.

Options:

• path (string, required) - Output file path
• pretty (boolean, optional) - Pretty print JSON (default: true)
• transform (function, optional) - Transform function (records) => transformedRecords
• encoding (string, optional) - File encoding (default: 'utf-8')

Returns: Promise<ExportResult>

interface ExportResult {
  path: string;   // Path where file was saved
  count: number;  // Number of records exported
  size: number;   // File size in bytes
}

Example:

import { exportJSON } from '@seedts/import';

const result = await exportJSON(userData, {
  path: './output/users.json',
  pretty: true,
  transform: (records) => records.map(r => ({
    ...r,
    password: undefined // Remove sensitive fields
  }))
});

console.log(`Exported ${result.count} records (${result.size} bytes)`);

exportCSV(data, options)

Export data to a CSV file.

Options: All from exportJSON plus:

• delimiter (string, optional) - Delimiter character (default: ',')
• header (boolean, optional) - Include headers (default: true)
• quoteChar (string, optional) - Quote character (default: '"')
• columns (string[], optional) - Columns to include (all if not specified)

Example:

import { exportCSV } from '@seedts/import';

await exportCSV(userData, {
  path: './output/users.csv',
  delimiter: ',',
  header: true,
  columns: ['id', 'name', 'email', 'createdAt']
});

exportYAML(data, options)

Export data to a YAML file. Same options as exportJSON (except pretty).

Example:

import { exportYAML } from '@seedts/import';

await exportYAML(configData, {
  path: './output/config.yaml'
});

exportData(data, options)

Generic export function that routes to the appropriate exporter based on format.

Example:

import { exportData } from '@seedts/import';

await exportData(userData, {
  path: './output/users.json',
  format: 'json',
  pretty: true
});

await exportData(userData, {
  path: './output/users.csv',
  format: 'csv',
  header: true
});

Merge Strategies

Combine imported data with existing data using different strategies.

mergeData(existingData, newData, options)

Strategies:

• replace - Replace all existing data with new data
• append - Add new data to existing data
• update - Update existing records that match, ignore new records
• upsert - Update existing records or insert new ones

Options:

• strategy (required) - Merge strategy to use
• keyField (string | string[]) - Key field(s) for matching (required for update/upsert)
• matchFn (function) - Custom match function (existing, new) => boolean

Examples:

import { mergeData } from '@seedts/import';

// Replace all data
const merged = mergeData(existingUsers, newUsers, {
  strategy: 'replace'
});

// Append new data
const merged = mergeData(existingUsers, newUsers, {
  strategy: 'append'
});

// Update by ID
const merged = mergeData(existingUsers, newUsers, {
  strategy: 'update',
  keyField: 'id'
});

// Upsert by email
const merged = mergeData(existingUsers, newUsers, {
  strategy: 'upsert',
  keyField: 'email'
});

// Custom match function
const merged = mergeData(existingUsers, newUsers, {
  strategy: 'upsert',
  matchFn: (existing, newRecord) => existing.email === newRecord.email
});

createMerger(options)

Create a reusable merge function for use in seed hooks.

Example:

import { createMerger, loadJSON } from '@seedts/import';

const upsertBySlug = createMerger({
  strategy: 'upsert',
  keyField: 'slug'
});

<Seed
  name="categories"
  adapter={adapter}
  beforeInsert={async (newData, ctx) => {
    const existing = await ctx.getSeed('categories') || [];
    return upsertBySlug(newData, existing);
  }}
>
  <Action>
    <Entity data={loadJSON('./data/categories.json')} />
  </Action>
</Seed>

Schema Validation

Use Zod schemas to validate imported data.

import { importJSON } from '@seedts/import';
import { z } from 'zod';

const productSchema = z.object({
  name: z.string().min(1),
  price: z.number().positive(),
  sku: z.string().regex(/^[A-Z0-9-]+$/),
  inStock: z.boolean(),
  category: z.enum(['electronics', 'clothing', 'books']),
  tags: z.array(z.string()).optional()
});

const result = await importJSON({
  path: './products.json',
  schema: productSchema,
  skipInvalid: true // Skip invalid records
});

console.log(`Valid: ${result.count}, Invalid: ${result.invalidCount}`);
result.errors.forEach(err => {
  console.log(`Record ${err.index}: ${err.error}`);
});

Complete Example

Data Files

users.json:

[
  {
    "name": "John Doe",
    "email": "[email protected]",
    "age": 30
  },
  {
    "name": "Jane Smith",
    "email": "[email protected]",
    "age": 25
  }
]

products.csv:

name,price,sku,inStock
"Laptop",999.99,LAP-001,true
"Mouse",29.99,MOU-002,true
"Keyboard",79.99,KEY-003,false

Seed Definition

import { Seed, Action, Entity, Attribute } from '@seedts/jsx-runtime';
import { loadJSON, loadCSV } from '@seedts/import';
import { z } from 'zod';

const userSchema = z.object({
  name: z.string(),
  email: z.string().email(),
  age: z.number().min(18)
});

export const UsersSeed = ({ adapter }) => (
  <Seed name="users" adapter={adapter}>
    <Action>
      <Entity data={loadJSON('./data/users.json')} />
    </Action>
  </Seed>
);

export const ProductsSeed = ({ adapter }) => (
  <Seed name="products" adapter={adapter}>
    <Action>
      <Entity data={loadCSV('./data/products.csv', {
        dynamicTyping: true
      })} />
    </Action>
  </Seed>
);

Export After Seeding

import { Executor } from '@seedts/core';
import { exportJSON, exportCSV } from '@seedts/import';

const executor = new Executor([UsersSeed, ProductsSeed]);
const results = await executor.execute();

// Export users to JSON
const usersData = results.find(r => r.name === 'users')?.data || [];
await exportJSON(usersData, {
  path: './output/users-seeded.json',
  pretty: true
});

// Export products to CSV
const productsData = results.find(r => r.name === 'products')?.data || [];
await exportCSV(productsData, {
  path: './output/products-seeded.csv',
  columns: ['id', 'name', 'price', 'sku']
});

Advanced Usage

Conditional Import with Merge

import { createMerger, loadJSON } from '@seedts/import';

const merger = createMerger({
  strategy: 'upsert',
  keyField: 'slug'
});

<Seed
  name="categories"
  adapter={adapter}
  condition={async (ctx) => {
    const existing = await ctx.getSeed('categories');
    return !existing || existing.length === 0;
  }}
  beforeInsert={async (newData, ctx) => {
    const existing = await adapter.query('categories') || [];
    return merger(newData, existing);
  }}
>
  <Action>
    <Entity data={loadJSON('./data/categories.json')} />
  </Action>
</Seed>

Transform During Import

import { importJSON } from '@seedts/import';

const result = await importJSON({
  path: './users.json',
  transform: async (record) => {
    // Add timestamps
    const transformed = {
      ...record,
      createdAt: new Date(),
      updatedAt: new Date()
    };

    // Hash password if present
    if (record.password) {
      transformed.password = await bcrypt.hash(record.password, 10);
    }

    // Normalize email
    transformed.email = record.email.toLowerCase().trim();

    return transformed;
  }
});

Batch Import Multiple Files

import { importJSON, mergeData } from '@seedts/import';

const files = [
  './data/users-batch-1.json',
  './data/users-batch-2.json',
  './data/users-batch-3.json'
];

let allUsers = [];

for (const file of files) {
  const result = await importJSON({ path: file });
  allUsers = mergeData(allUsers, result.data, {
    strategy: 'upsert',
    keyField: 'email'
  });
}

console.log(`Total unique users: ${allUsers.length}`);

TypeScript Support

All functions are fully typed:

import { importJSON, ImportResult } from '@seedts/import';
import { z } from 'zod';

interface User {
  name: string;
  email: string;
  age: number;
}

const userSchema = z.object({
  name: z.string(),
  email: z.string().email(),
  age: z.number()
});

// Fully typed result
const result: ImportResult<User> = await importJSON<User>({
  path: './users.json',
  schema: userSchema
});

// result.data is User[]
result.data.forEach((user: User) => {
  console.log(user.name, user.email);
});

Error Handling

import { importJSON } from '@seedts/import';

try {
  const result = await importJSON({
    path: './data.json',
    schema: mySchema,
    skipInvalid: true
  });

  if (result.invalidCount > 0) {
    console.warn(`${result.invalidCount} invalid records skipped`);
    result.errors.forEach(err => {
      console.error(`Record ${err.index}:`, err.error);
    });
  }

  console.log(`Successfully imported ${result.count} records`);
} catch (error) {
  console.error('Import failed:', error);
}

License

MIT