file-parser

A library for parsing data files.

Currently it only parses commas delimited data file, but potentially extensible with other parsers.

Contents

• Usage
  • CSV Parser
    • CSV Schema
• Types of Tests Running with Jest

Usage

npm i @dcube-engineering/file-parser

CSV Parser

import { parseFile, ParseConfig, InputFormat, OutputFormat } from '@dcube-engineering/file-parser';

// create a parser input parameters
const param = {
    inputFilePath: '<input-file-path>',
    inputFileFormat: InputFormat.CSV,
    outputFilePath: '<output-file-path>',
    outputFileFormat: OutputFormat.JSON,
    options: {
        skipHeader: '<true|false>',
        schemaPath: '<schema-file-path>',
        skipError: '<true|false>'
    }
};

await parseFile(param as ParseConfig);

parseFile method required ParseConfig type as input parameter.

ParseConfig consists of:

• inputFilePath: The file path for the raw input file
• inputFileFormat: The file type for the input file. (eg. InputFormat.CSV)
• outputFilePath: The file path for the output file
• outputFileFormat: The file type for the output file. (eg. OutputFormat.JSON)
• options:
  • skipHeader: (Optional) boolean indicator to skip the first line of the input file. If skipHeader is undefined, the first line of the file will not be skipped. (default to false)
  • schemaPath: (Mandatory) The file path of the schema validation file. Mandatory for csv input file type.
  • skipError: (Optional) boolean indicator to skip rows with validation errors and continue processing. If skipError is undefined, the error will be thrown and processing will not continue. (default to false)

CSV Schema

The schema is simply an array of field specifications, indicating how fields should be extracted from each line of the input stream. These are the properties for the field specifications:

• index: (Mandatory) The position of the field to be retrieved from each line of the input stream.
• header: (Mandatory) Name of the field. This will be the property name used in output file.
• validator: (Optional) An object specifying validators to be used to validate the field. Attribute keys (email/regex) indicate the type of validator, and attribute values provide parameters to the validator. If any validator fails, a validation error will be returned. If no validators are specified, then no validation will be done.
• validateNonNullOnly: (Optional) Whether a field value should be validated, if it is empty. If true, then an empty field value (i.e. empty string after trimming) will not be processed by validators (i.e. will not fail any validations). If validateNonNullOnly is undefined, then validation will be done.

Example:

[
    {
        "index": 1,
        "header": "country"
    },
    {
        "index": 2,
        "header": "city",
        "validateNonNullOnly": true
    },
    {
        "index": 3,
        "header": "email",
        "validator": {
            "email": true
        }
    }
]

Types of Tests Running with Jest

1. Unit

   npm t or npm run test

   • To run individual test file

   npm t <absolute-path>