CSV Split

CLI utility to split CSV into multiples files base on value of a specified column.

Feature

You have one or more data sheets. You want to split data into files and each one groups the records which have same value in a column.

input.csv

| id | name | group | |----|------|-------| | 1 | Alice | A | | 2 | Bob | B | | 3 | Charley | A | | 4 | David | B |

Being processed with csvsplit, base on value of column group, it becomes:

A.csv

| id | name | group | |----|------|-------| | 1 | Alice | A | | 3 | Charley | A |

B.csv

| id | name | group | |----|------|-------| | 2 | Bob | B | | 4 | David | B |

It's able to process multiple input files. No need to combile them beforehead.

Install

This tool runs on node.js execution environment. If node.js hasn't been installed on your OS, download from Node.js official site and install it.

On command line, type node --version to make sure node.js has been installed properly. Node.js v12.0 or higher is required.

Install CSV split:

npm install -g @aleung/csvsplit

It runs on any OS platform that node.js supports.

You could run CSV split without installation by adding npx before the command:

npx csvsplit [options...]

Usage

$ csvsplit --help
Usage: /usr/local/bin/csvsplit [options] <inputfiles>

Options:
  -c, --column  The index of column base which to split (A=1, B=2 ...)             [number] [required]
  -f, --filter  If set, only when the column value is in this list will be outputed. Configure with a
                comma seperated list of values.                                               [string]
  -o, --output  Directory to put generated files into                         [string] [default: "./"]
  --no-header  The input CSV file has no header                             [boolean] [default: false]
  --help        Show help                                                                    [boolean]
  --version     Show version number                                                          [boolean]

Examples:
  /usr/local/bin/csvsplit -c 2 -o ./output             Process all CSV files and split base on the 2nd
  ./input/*.csv                                        column

The generated csv files are named into the value of the column base on which to split.

There might be an extra csv file generated, in which contains only the header. Just delete it.

If you get "Too many open files" issue, try to use --filter option to limit the number of generated files.

Development

$ yarn install
$ yarn test