Dao.js

Node.js command-line utility for fast data processing. It is designed to help in analyzing and operating with the large datasets.

Installation

$ npm install -g dao.js

Examples

Display help:

$ dao --help
  usage: dao [method, ...] [options]
 
  options:
    -i, --input        Input data file
    -o, --output       Output data file, default 'o.json'
    -e, --external     External module with extra methods
    -v, --version      Show version
    -h, --help         You're staring at it

  methods:
    create [filename]            Create an empty file
    clear [filename]             Clear file content
    copy [filename1 filename2]   Copy 'filename1' -> 'filename2', create 'filename2' if not exists
    remove [filename]            Remove file

An example of basic manipulations with files:

$ dao create data.json
$ dao clear data.json
$ dao copy data.json newdata.json
$ dao remove data.json

Default file for output is o.json.

Operations with data:

$ dao method1 -i data.json                      # default output file is 'o.json'
$ dao method1 -i data.json -o newData.json      # specify output file via option '-o' 

with this command we apply method1 for our data in data.json and result in o.json by default.

We can include any external module with custom methods, and then can use it from command-line via -e option.

For example, lets apply custom filter for our data.json:

[
    { "id": 1, "value": 3},
    { "id": 2, "value": 5},
    { "id": 3, "value": 1},
    { "id": 4, "value": 2},
    { "id": 5, "value": 6}
]

Create auxiliary file utils.js with custom methods (in the same directory as data.json):

exports.customFilter = function(data){
    // data - is our data that we want to process
	return data.filter(function(d){
	    return d.value < 3;
	});	
};

And then just type:

$ dao customFilter -e utils.js -i data.json

it results in o.json:

[
    { "id": 3, "value": 1},
    { "id": 4, "value": 2}
]

License

MIT