@wuyuchentr/file-splitter

Split large files by line count or size. Zero dependencies, stream-based, memory-friendly.

Read a single line at a time — works with files of any size.

Install

npm install @wuyuchentr/file-splitter

Usage

const { splitFile } = require('@wuyuchentr/file-splitter');

// Split by lines
const files = await splitFile('large.log', { linesPerFile: 10000 });
// → ['large-1.log', 'large-2.log', ...]

// Split by approximate size (respects line boundaries)
const files = await splitFile('data.csv', { sizePerFile: 1024 * 1024 });
// → ['data-1.csv', 'data-2.csv', ...]

Options

| Option | Default | Description | |--------|---------|-------------| | linesPerFile | — | Max lines per output file | | sizePerFile | — | Approximate bytes per output file (boundary-safe) | | outputDir | input dir | Directory for output files | | outputPrefix | basename | Custom prefix before -1.ext | | encoding | 'utf-8' | File encoding |

How it works

Uses readline + fs.createReadStream under the hood. Reads one line at a time, writes to sequential output files. Memory usage stays constant regardless of file size.