@bernierllc/csv-import-suite

Complete CSV import solution combining csv-import service and csv-ui components with orchestration, error handling, and user-friendly workflows.

Installation

npm install @bernierllc/csv-import-suite

Features

• Complete Import Workflow - End-to-end CSV import with parsing, validation, mapping, and processing
• React Components - Pre-built UI components for import wizard and management interface
• Error Handling - Comprehensive error handling with suggestions and recovery workflows
• Schema Management - Dynamic schema creation, validation, and reuse
• Progress Tracking - Real-time import progress with status updates and time estimation
• Data Preview - Preview and validate data before importing
• Accessibility - WCAG 2.1 AA compliant with full keyboard navigation and screen reader support
• Storage Options - Multiple storage backends (memory, localStorage, sessionStorage, custom)
• TypeScript - Full TypeScript support with comprehensive type definitions

Quick Start

Basic Import

import { CSVImportSuite } from '@bernierllc/csv-import-suite';

const suite = new CSVImportSuite({
  csvImport: {
    maxFileSize: '10MB',
    allowedTypes: ['.csv']
  },
  ui: {
    theme: 'light',
    accessibility: {
      announceProgress: true
    }
  }
});

// Import CSV file
const result = await suite.importFile(file, {
  schema: {
    id: 'users',
    name: 'User Import',
    fields: [
      { name: 'name', type: 'string', required: true },
      { name: 'email', type: 'email', required: true },
      { name: 'age', type: 'number', required: false }
    ],
    validationRules: [],
    businessRules: []
  },
  callbacks: {
    onProgress: (progress) => console.log(`Progress: ${progress.progress}%`),
    onComplete: (result) => console.log('Import completed:', result)
  }
});

console.log(`Imported ${result.validRows} valid rows`);

React Components

import { ImportWizard, ImportManager } from '@bernierllc/csv-import-suite';

// Complete import wizard
function ImportPage() {
  const handleComplete = (result) => {
    console.log('Import completed:', result);
  };

  return (
    <ImportWizard
      onComplete={handleComplete}
      onCancel={() => console.log('Cancelled')}
      initialSchema={{
        fields: [
          { name: 'name', type: 'string', required: true },
          { name: 'email', type: 'email', required: true }
        ]
      }}
    />
  );
}

// Import management interface
function ManagePage() {
  return (
    <ImportManager
      onImportSelect={(id) => console.log('Selected:', id)}
      onImportDelete={(id) => console.log('Deleted:', id)}
      onImportRetry={(id) => console.log('Retrying:', id)}
    />
  );
}

Schema Management

// Create reusable schema
const schemaResult = await suite.createSchema({
  name: 'User Import Schema',
  description: 'Schema for importing user data',
  fields: [
    {
      name: 'email',
      type: 'email',
      required: true,
      validation: {
        pattern: /^[^\s@]+@[^\s@]+\.[^\s@]+$/
      }
    },
    {
      name: 'firstName',
      type: 'string',
      required: true,
      validation: {
        minLength: 1,
        maxLength: 50
      }
    },
    {
      name: 'age',
      type: 'number',
      required: false,
      validation: {
        min: 0,
        max: 150
      }
    }
  ],
  validationRules: [
    {
      type: 'required',
      field: 'email',
      message: 'Email is required',
      severity: 'error'
    }
  ]
});

// Use schema for import
const result = await suite.importFile(file, {
  schemaId: schemaResult.schemaId
});

API Reference

CSVImportSuite

Main orchestration class that coordinates all import operations.

Constructor

new CSVImportSuite(config?: CSVImportSuiteConfig)

Core Methods

importFile(file: File, config: ImportConfig): Promise<ImportResult>

Import CSV file with full orchestration.

const result = await suite.importFile(file, {
  schema: importSchema,
  options: {
    batchSize: 100,
    validateOnly: false
  },
  callbacks: {
    onProgress: (progress) => console.log(progress),
    onComplete: (result) => console.log(result),
    onError: (error) => console.error(error)
  }
});

importFromUrl(url: string, config: ImportConfig): Promise<ImportResult>

Import CSV from URL.

const result = await suite.importFromUrl('https://example.com/data.csv', {
  schema: importSchema
});

importFromString(content: string, config: ImportConfig): Promise<ImportResult>

Import CSV from string content.

const csvContent = 'name,email\nJohn,[email protected]';
const result = await suite.importFromString(csvContent, {
  schema: importSchema
});

previewData(file: File, schema?: ImportSchema): Promise<PreviewResult>

Preview data without importing.

const preview = await suite.previewData(file, schema);
console.log('Headers:', preview.headers);
console.log('Sample data:', preview.rows);
console.log('Validation errors:', preview.errors);

Schema Management

createSchema(schema: SchemaData): Promise<SchemaResult>

Create new import schema.

const result = await suite.createSchema({
  name: 'Product Import',
  fields: [
    { name: 'sku', type: 'string', required: true },
    { name: 'name', type: 'string', required: true },
    { name: 'price', type: 'number', required: true }
  ]
});

listSchemas(options?: ListOptions): Promise<SchemaList>

List available schemas.

const schemas = await suite.listSchemas({
  page: 1,
  pageSize: 10,
  sortBy: 'name',
  sortOrder: 'asc'
});

Import Management

getImportStatus(importId: string): Promise<ImportStatus>

Get current import status.

const status = await suite.getImportStatus(importId);
console.log(`Status: ${status.status}, Progress: ${status.progress}%`);

getImportHistory(options?: HistoryOptions): Promise<ImportHistory>

Get import history.

const history = await suite.getImportHistory({
  status: 'completed',
  limit: 20
});

cancelImport(importId: string): Promise<CancelResult>

Cancel active import.

const result = await suite.cancelImport(importId);

React Components

ImportWizard

Complete step-by-step import workflow.

<ImportWizard
  onComplete={(result) => console.log(result)}
  onCancel={() => console.log('cancelled')}
  onError={(error) => console.error(error)}
  initialSchema={schema}
  config={suiteConfig}
  className="my-wizard"
  theme="dark"
/>

Props:

• onComplete: (result: ImportResult) => void - Called when import completes
• onCancel?: () => void - Called when user cancels
• onError?: (error: Error) => void - Called on error
• initialSchema?: ImportSchema - Pre-populate schema
• config?: CSVImportSuiteConfig - Suite configuration
• className?: string - CSS class name
• theme?: 'light' | 'dark' - UI theme

ImportManager

Import history and management interface.

<ImportManager
  onImportSelect={(id) => viewImport(id)}
  onImportDelete={(id) => deleteImport(id)}
  onImportRetry={(id) => retryImport(id)}
  config={suiteConfig}
  className="my-manager"
/>

Props:

• onImportSelect?: (importId: string) => void - Called when import selected
• onImportDelete?: (importId: string) => void - Called when import deleted
• onImportRetry?: (importId: string) => void - Called when import retried
• config?: CSVImportSuiteConfig - Suite configuration
• className?: string - CSS class name

SchemaEditor

Create and edit import schemas.

<SchemaEditor
  schema={currentSchema}
  onSchemaChange={setSchema}
  onSave={saveSchema}
  onCancel={cancelEdit}
  readOnly={false}
  className="my-editor"
/>

Configuration

CSVImportSuiteConfig

interface CSVImportSuiteConfig {
  csvImport?: {
    maxFileSize?: string | number;     // '10MB' or bytes
    allowedTypes?: string[];           // ['.csv', 'text/csv']
    chunkSize?: number;                // 1000
    timeout?: number;                  // 300000ms (5 minutes)
  };
  ui?: {
    theme?: 'light' | 'dark' | 'auto';
    language?: string;                 // 'en'
    accessibility?: {
      announceProgress?: boolean;      // true
      highContrast?: boolean;          // false
      focusManagement?: boolean;       // true
      screenReaderOptimized?: boolean; // true
    };
  };
  storage?: {
    provider?: 'memory' | 'localStorage' | 'sessionStorage' | 'custom';
    maxStorageSize?: number;           // 50MB in bytes
    customProvider?: StorageProvider;
  };
  validation?: {
    enableClientSideValidation?: boolean; // true
    strictMode?: boolean;                  // false
    customValidators?: Record<string, Function>;
  };
  performance?: {
    batchSize?: number;                // 100
    maxConcurrentOperations?: number;  // 3
    enableProgressTracking?: boolean;  // true
    debounceMs?: number;               // 300
  };
}

Default Configuration

import { createDefaultConfig } from '@bernierllc/csv-import-suite';

const config = createDefaultConfig();
const suite = new CSVImportSuite(config);

Field Types

Supported field types for schema definitions:

• 'string' - Text values
• 'number' - Numeric values (integers and decimals)
• 'boolean' - Boolean values (true/false, 1/0, yes/no)
• 'date' - Date values (various formats supported)
• 'email' - Email addresses with validation
• 'url' - Web URLs with validation
• 'phone' - Phone numbers (basic validation)
• 'custom' - Custom validation with user-defined rules

Validation Rules

Field-Level Validation

{
  name: 'email',
  type: 'email',
  required: true,
  validation: {
    pattern: /^[^\s@]+@[^\s@]+\.[^\s@]+$/,
    minLength: 5,
    maxLength: 100
  }
}

Schema-Level Validation Rules

validationRules: [
  {
    type: 'required',
    field: 'email',
    message: 'Email is required',
    severity: 'error'
  },
  {
    type: 'unique',
    field: 'id',
    message: 'ID must be unique',
    severity: 'error'
  }
]

Business Rules

businessRules: [
  {
    id: 'age-validation',
    name: 'Age Validation',
    type: 'custom',
    validator: (row, allRows) => {
      const age = parseInt(row[2]);
      if (age < 0 || age > 150) {
        return {
          success: false,
          error: {
            message: 'Age must be between 0 and 150',
            field: 'age',
            severity: 'error'
          }
        };
      }
      return { success: true };
    }
  }
]

Error Handling

Error Types

• Validation Errors - Data doesn't match schema requirements
• Parsing Errors - CSV format issues
• Mapping Errors - Column mapping problems
• Business Rule Errors - Custom business logic violations
• System Errors - File access, memory, or processing errors

Error Recovery

const result = await suite.importFile(file, {
  schema,
  callbacks: {
    onError: (error) => {
      console.error('Import error:', error.message);
      
      // Get suggestions for common errors
      if (error.code === 'INVALID_EMAIL') {
        console.log('Suggestion:', error.suggestion);
      }
    }
  }
});

// Check for errors in result
if (!result.success) {
  console.log('Import completed with errors:');
  result.errors.forEach(error => {
    console.log(`Row ${error.row}: ${error.message}`);
    if (error.suggestion) {
      console.log(`Suggestion: ${error.suggestion}`);
    }
  });
}

Accessibility

The CSV Import Suite is designed with accessibility as a core requirement:

WCAG 2.1 AA Compliance

• Keyboard Navigation - All components fully navigable by keyboard
• Screen Reader Support - Complete ARIA labeling and live regions
• Focus Management - Logical tab order and focus indicators
• Color Contrast - 4.5:1 minimum contrast ratio for all text
• Progress Announcements - Real-time progress updates for assistive technology

Screen Reader Features

const suite = new CSVImportSuite({
  ui: {
    accessibility: {
      announceProgress: true,        // Announce progress changes
      screenReaderOptimized: true,   // Optimize for screen readers
      focusManagement: true          // Manage focus automatically
    }
  }
});

Keyboard Shortcuts

• Tab/Shift+Tab - Navigate between elements
• Enter/Space - Activate buttons and controls
• Escape - Cancel operations or close dialogs
• Arrow Keys - Navigate within lists and grids

Performance

Optimization Features

• Chunked Processing - Large files processed in manageable chunks
• Background Processing - Non-blocking import operations
• Progress Tracking - Real-time progress with time estimation
• Memory Management - Efficient memory usage for large datasets
• Caching - Cache parsed data and validation results

Performance Benchmarks

• Small files (< 1MB): < 5s processing time
• Medium files (1-10MB): < 30s processing time
• Large files (> 10MB): < 2min processing time
• Memory usage: < 50MB for 1GB file
• UI responsiveness: < 100ms for user interactions

Performance Configuration

const suite = new CSVImportSuite({
  performance: {
    batchSize: 1000,              // Rows per batch
    maxConcurrentOperations: 5,   // Concurrent operations
    debounceMs: 200              // UI update debouncing
  }
});

Storage Options

Memory Storage (Default)

const suite = new CSVImportSuite({
  storage: {
    provider: 'memory'
  }
});

Browser Storage

// LocalStorage - persists across sessions
const suite = new CSVImportSuite({
  storage: {
    provider: 'localStorage',
    maxStorageSize: 10 * 1024 * 1024 // 10MB
  }
});

// SessionStorage - persists within session
const suite = new CSVImportSuite({
  storage: {
    provider: 'sessionStorage'
  }
});

Custom Storage

class DatabaseStorageProvider implements StorageProvider {
  async store(key: string, data: any): Promise<void> {
    await database.save(key, data);
  }

  async retrieve(key: string): Promise<any> {
    return await database.load(key);
  }

  async remove(key: string): Promise<void> {
    await database.delete(key);
  }

  async clear(): Promise<void> {
    await database.clear();
  }
}

const suite = new CSVImportSuite({
  storage: {
    provider: 'custom',
    customProvider: new DatabaseStorageProvider()
  }
});

Examples

Complete Implementation

import React, { useState } from 'react';
import { 
  CSVImportSuite, 
  ImportWizard, 
  ImportManager,
  ImportResult 
} from '@bernierllc/csv-import-suite';

function CSVImportApp() {
  const [view, setView] = useState<'import' | 'manage'>('import');
  const [results, setResults] = useState<ImportResult[]>([]);

  const suite = new CSVImportSuite({
    csvImport: {
      maxFileSize: '50MB',
      allowedTypes: ['.csv', 'text/csv']
    },
    ui: {
      theme: 'light',
      accessibility: {
        announceProgress: true,
        screenReaderOptimized: true
      }
    },
    storage: {
      provider: 'localStorage'
    }
  });

  const handleImportComplete = (result: ImportResult) => {
    setResults(prev => [result, ...prev]);
    console.log('Import completed:', result);
    
    if (result.success) {
      alert(`Successfully imported ${result.validRows} rows!`);
    } else {
      alert(`Import completed with ${result.errors.length} errors`);
    }
  };

  const handleImportError = (error: Error) => {
    console.error('Import error:', error);
    alert(`Import failed: ${error.message}`);
  };

  return (
    <div className="csv-import-app">
      <nav>
        <button 
          onClick={() => setView('import')}
          className={view === 'import' ? 'active' : ''}
        >
          Import CSV
        </button>
        <button 
          onClick={() => setView('manage')}
          className={view === 'manage' ? 'active' : ''}
        >
          Manage Imports
        </button>
      </nav>

      {view === 'import' ? (
        <ImportWizard
          onComplete={handleImportComplete}
          onError={handleImportError}
          initialSchema={{
            fields: [
              { name: 'name', type: 'string', required: true },
              { name: 'email', type: 'email', required: true },
              { name: 'phone', type: 'phone', required: false }
            ]
          }}
        />
      ) : (
        <ImportManager
          onImportSelect={(id) => console.log('Selected:', id)}
          onImportRetry={(id) => console.log('Retrying:', id)}
        />
      )}
    </div>
  );
}

export default CSVImportApp;

Dependencies

This package orchestrates several BernierLLC packages:

• @bernierllc/csv-import - Core CSV import functionality
• @bernierllc/csv-ui - React UI components
• @bernierllc/retry-policy - Retry logic and error handling

Peer Dependencies

• react (^18.0.0) - For UI components
• react-dom (^18.0.0) - For UI components

Contributing

Please read our contributing guidelines and ensure all tests pass before submitting pull requests.

# Install dependencies
npm install

# Run tests
npm test

# Run tests with coverage
npm run test:coverage

# Build package
npm run build

# Lint code
npm run lint

License

Copyright (c) 2025 Bernier LLC. All rights reserved.

This package is licensed under a limited-use license. See LICENSE file for details.