📄 readline-pager

⚡ Memory-efficient, paginated file reader for Node.js with async iteration, prefetching, backward reading and optional worker support. readline-pager reads large text files page-by-page without loading the entire file into memory.

• ✅ Zero dependencies
• ✅ Async iterator (for await...of) + manual next() API
• ✅ Forward & backward reading (EOF → BOF)
• ✅ Optional worker thread mode (forward only)
• ✅ Up to ~3x faster than Node.js readline
• ✅ ~97% test coverage & fully typed (TypeScript)

Important: Performance is heavily dependent on the chunkSize option; ensure you fine-tune it for your specific I/O hardware. A setting of 64 KB is typically a good starting point. Increasing it might gradually improve read speeds, usually reaching an optimal peak depending on your hardware's capabilities.

📦 Installation

npm install readline-pager

🚀 Quick start

import { createPager } from "readline-pager";
// const { createPager } = require("readline-pager");

const pager = createPager("./bigfile.txt");

for await (const page of pager) {
  console.log(page[0]); // first line of the current page
}

Recommended for highest throughput:

while (true) {
  const page = await pager.next();
  if (!page) break;
}

// or
let page;
while ((page = await pager.next()) !== null) {}

• while + next() is the fastest iteration method (avoids extra async-iterator overhead).
• for await of is more ergonomic and convenient.

⚙️ Options

createPager(filepath, {
  chunkSize?: number,     // default: 64 * 1024 (64 KiB)
  pageSize?: number,      // default: 1_000
  delimiter?: string,     // default: "\n"
  prefetch?: number,      // default: 1
  backward?: boolean,     // default: false
  useWorker?: boolean,    // default: false (forward only)
});

• chunkSize — number of bytes read per I/O operation.
• pageSize — number of lines per page.
• delimiter — line separator.
• prefetch — max number of pages buffered internally. Not required for typical use; tuning has little effect once the engine is optimized.
• backward — read file from end → start (not supported with useWorker).
• useWorker — offload parsing to a worker thread (forward only).

📚 API

pager.next(): Promise<string[] | null>

Returns the next page or null when finished. Empty lines are preserved.

Note: Unlike Node.js readline, which may skip empty files or leading empty lines, readline-pager always returns all lines.

• A completely empty file (0 bytes) produces [""] on the first read.
• A file with multiple empty lines returns each line as an empty string (e.g., ["", ""] for two empty lines). Node.js readline may emit fewer or no line events in these cases.

pager.close(): Promise<void>

Stops reading and releases resources immediately. Safe to call at any time.

Read-only properties

• pager.lineCount — lines emitted so far
• pager.firstLine — first emitted line (available after first read)
• pager.lastLine — last emitted line (updated per page)

📊 Benchmark

Run the included benchmark:

# default run
npm run benchmark

# or customize with args
node test/_benchmark.ts --lines=20000 --page-size=500 --backward

Test setup: generated text files with uuid, run on a fast NVMe machine with default options; values are averages from multiple runs. Results are machine-dependent.

The Average Throughput (MB/s) is computed for two strategies: reading files line by line and page by page.

In addition to Node, the two other popular JavaScript runtimes were also tested with readline-pager.

Line by line

| Runtime / Method | 1M lines (35 MB) | 10M lines (353 MB) | 100M lines (3,529 MB) | 1,000M lines (35,286 MB) | | ---------------- | ---------------: | -----------------: | --------------------: | -----------------------: | | Node — node:line | 369 | 435 | 455 | 455 | | Deno — node:line | 203 | 230 | 230 | 229 | | Deno — deno:line | 738 | 901 | 915 | 809 | | Bun — node:line | 246 | 279 | 283 | 280 | | Bun — bun:line | 938 | 1,540 | 1,668 | 1,315 |

Page by page

| Runtime / Method | 1M lines (35 MB) | 10M lines (353 MB) | 100M lines (3,529 MB) | 1,000M lines (35,286 MB) | | --------------------- | ---------------: | -----------------: | --------------------: | -----------------------: | | Node — readline-pager | 1,053 | 1,311 | 1,278 | 936 | | Deno — deno:page | 852 | 909 | 908 | 783 | | Deno — readline-pager | 1,131 | 1,268 | 1,271 | 911 | | Bun — bun:page | 411 | 440 | 449 | 428 | | Bun — readline-pager | 827 | 1,021 | 1,040 | 804 |

Runtime Environment: Node.js v25.6.1 & Bun v1.3.9 & Deno 2.6.10

🛠 Development & Contributing

• Minimum supported Node.js: v18.12 (lts/hydrogen).
• Development/test environment: Node v25.6 & TypeScript v5.9.

Run tests:

npm ci
npm test

Contributions are welcome — feel free to open an issue or PR.

📜 License

MIT — © Morteza Jamshidi