untar-csv
v1.1.1
Published
Read a tar of multiple csv files as a stream of objects
Readme
Usage
const fs = require ('fs')
const zlib = require ('zlib')
const {TarCsvReader} = require ('untar-csv')
const reader = TarCsvReader ({
// test: entry => entry.name.endsWith ('.csv'),
// delimiter: ',',
// skip: 0, // header lines
// fileNumField: '#', // how to name the file # property
// rowNumField: '##', // how to name the line # property
// empty: null,
// recordClass: Array // to get [id, name] instead of {id, name}
// columnClass: class extends CSVColumn {...}
columns: ['id', 'name'],
// header: 2, // to read `columns` from first two CSV lines
// mask: 0b101, // only the 1st and 3rd column will be read
// maxColumns: 16384, // leak protection: 16384 is the Excel limit
})
fs.createReadStream ('lots-of-data.tar.gz').pipe (zlib.createGunzip ()).pipe (reader)
for await (const {id, name} of reader) {
// do something with `id` and `name`
}Options
Most options are effectively passed to CSVReader, see there for details.
|Name|Default value|Description|
|-|-|-|
|columnClass|CSVColumn||
|columns| |Array of explicit column definitions (mutually exclusive with the header option)|
|delimiter|','|Column delimiter|
|empty|null|The value corresponding to zero length cell content|
|fileNumField | null | The name of the file # property (null for no numbering)
|header|0|Number of lines to gather columns definition|
|mask|0|Unless 0 with header set, only masked columns will be read|
|maxColumns|16384|Maximum # of columns in the header (to prevent a memory overflow)|
|recordClass|Object|May be set as Array|
|rowNumField | null | The name of the line # property (null for no numbering)
|skip|0|Number of top lines to ignore (prior to header)|
|test| ({name}) => true |tar entry filter, structure described at tar-stream|
