@bernierllc/csv-mapper

Intelligent column mapping functionality for CSV data with field variations, fuzzy auto-mapping via Jaro-Winkler/Levenshtein scoring, type conversion, and extended schema field metadata.

Features

• Column Mapping: Map CSV columns to structured field names
• Field Variations: Handle multiple possible column names for the same field
• Fuzzy Auto-Mapping: When no exact variation matches, falls back to Jaro-Winkler + Levenshtein scoring via @bernierllc/string-similarity for headers that don't match any known variation
• Schema Metadata Fields: label, group, source, identity, mergeable — used by entity resolution and merge planning downstream
• Type Conversion: Convert string data to appropriate types
• Mapping Validation: Validate mapping configurations
• Bulk Operations: Process entire datasets with mapping rules

Installation

npm install @bernierllc/csv-mapper

Quick Start

Basic Column Mapping

import { CSVMapper, FieldType } from '@bernierllc/csv-mapper';

const mapper = new CSVMapper();

const mapping = {
  'email_address': { targetField: 'email', type: FieldType.EMAIL },
  'first_name': { targetField: 'firstName', type: FieldType.STRING },
  'last_name': { targetField: 'lastName', type: FieldType.STRING },
  'phone': { targetField: 'phone', type: FieldType.PHONE },
  'dob': { targetField: 'dateOfBirth', type: FieldType.DATE }
};

const csvRow = ['[email protected]', 'John', 'Doe', '555-1234', '1990-01-01'];
const mappedRow = mapper.mapRow(csvRow, mapping);

// Result: {
//   email: '[email protected]',
//   firstName: 'John',
//   lastName: 'Doe',
//   phone: '5551234',
//   dateOfBirth: Date('1990-01-01')
// }

Auto-Mapping (Exact + Fuzzy)

const headers = ['email_address', 'first_name', 'last_name', 'phone_number'];
const targetFields = ['email', 'firstName', 'lastName', 'phone'];

// Generate mapping suggestions
const suggestions = mapper.suggestMapping(headers, targetFields);
console.log(suggestions);
// [
//   { csvColumn: 'email_address', targetField: 'email', confidence: 0.95, reasoning: 'Field variation: ...' },
//   { csvColumn: 'first_name', targetField: 'firstName', confidence: 0.95, reasoning: 'Field variation: ...' },
//   ...
// ]

// Auto-generate mapping (uses fuzzy fallback for headers with no variation match)
const autoMapping = mapper.autoMap(headers, targetFields);

Fuzzy matching details

When no known field variation matches a CSV header, the auto-mapper falls back to a blended Jaro-Winkler / Levenshtein score from @bernierllc/string-similarity. Fuzzy scores are:

• capped at 0.94 — always below a variation match (0.95) at equal confidence, so they never displace a known-variation match in the ranked output.
• filtered by confidenceThreshold (default 0.7) — low-quality fuzzy matches are excluded.
• identified in MappingSuggestion.reasoning — the string contains "Fuzzy match" and the algorithm names so consumers can distinguish fuzzy from exact/variation hits.

const suggestions = AutoMapper.suggestMapping(['eml_addr'], ['email']);
// [{ csvColumn: 'eml_addr', targetField: 'email', confidence: ~0.80,
//    reasoning: 'Fuzzy match: "eml_addr" is 80% similar to "email" (Jaro-Winkler/Levenshtein)' }]

Bulk Processing

const csvRows = [
  ['[email protected]', 'John', 'Doe', '555-1234'],
  ['[email protected]', 'Jane', 'Smith', '555-5678']
];

const mappedRows = mapper.mapRows(csvRows, mapping);

API Reference

CSVMapper

Main class for CSV mapping operations.

Constructor

new CSVMapper(config?: MapperConfig)

Methods

• mapRow(row: string[], mapping: ColumnMapping): MappedRow
• mapRows(rows: string[][], mapping: ColumnMapping): MappedRow[]
• suggestMapping(headers: string[], targetFields: string[]): MappingSuggestion[]
• autoMap(headers: string[], targetFields: string[]): ColumnMapping
• validateMapping(mapping: ColumnMapping, headers: string[]): ValidationResult

Schema Fields (FieldMapping)

The FieldMapping interface accepts these fields (all optional except targetField):

| Field | Type | Description | |---|---|---| | targetField | string | Required. Destination field name in the mapped output. | | type | FieldType | Field type for type conversion. | | required | boolean | Whether the field must be present in the CSV. | | defaultValue | any | Value used when the column is missing. | | transform | (value: string) => any | Custom transform applied before type conversion. | | validation | ValidationRule[] | Validation rules for the field. | | label | string | Human-readable label (used in UI and conflict resolution). | | group | string | Logical group for organizing related fields (e.g. 'contact', 'address'). | | source | 'core' \| 'custom' | Whether this field is from the core schema or added at runtime. | | identity | boolean | When true, used for entity matching; never treated as a conflict. | | mergeable | boolean | When true, differing values between incoming and existing records are surfaced as conflicts. |

label, group, source, identity, and mergeable are consumed by downstream packages (@bernierllc/merge-planner, @bernierllc/entity-resolver) and are ignored by the mapper itself.

Field Types

Supported field types for data conversion:

• FieldType.STRING - String values
• FieldType.NUMBER - Numeric values
• FieldType.BOOLEAN - Boolean values
• FieldType.DATE - Date values
• FieldType.EMAIL - Email addresses
• FieldType.PHONE - Phone numbers
• FieldType.URL - URLs
• FieldType.JSON - JSON data

Field Variations

Built-in field variations handle common CSV column name variations:

• email → ['email', 'e-mail', 'email_address', 'emailaddress', 'mail']
• firstName → ['first_name', 'firstname', 'first name', 'fname', 'given_name']
• lastName → ['last_name', 'lastname', 'last name', 'lname', 'family_name']
• And many more...

Advanced Usage

Custom Field Variations

import { FieldVariationManager } from '@bernierllc/csv-mapper';

// Add custom variations
FieldVariationManager.addVariations('customField', ['custom_field', 'customfield', 'cf']);

// Get variations for a field
const variations = FieldVariationManager.getVariations('email');

Type Conversion

import { TypeConverter, FieldType } from '@bernierllc/csv-mapper';

// Convert values
const result = TypeConverter.convert('[email protected]', FieldType.EMAIL);
console.log(result); // { value: '[email protected]', isValid: true }

// Infer types
const inferredType = TypeConverter.inferType('123.45');
console.log(inferredType); // FieldType.NUMBER

Configuration

const mapper = new CSVMapper({
  caseSensitive: false,
  fuzzyMatching: true,
  fuzzyThreshold: 0.7,
  enableAutoMapping: true,
  strictMode: false
});

Error Handling

The mapper provides detailed error information:

const mappedRow = mapper.mapRow(csvRow, mapping);

if (mappedRow._mappingErrors) {
  console.log('Mapping errors:', mappedRow._mappingErrors);
  // [
  //   {
  //     column: 'email_address',
  //     field: 'email',
  //     error: 'Invalid email format',
  //     value: 'invalid-email'
  //   }
  // ]
}

Performance

Optimized for large datasets:

• Small files (< 1MB): < 100ms processing time
• Medium files (1-10MB): < 1s processing time
• Large files (> 10MB): < 10s processing time

License

Bernier LLC - All rights reserved.