@padmaj/batchify

Process large arrays in batches with concurrency control. Zero dependencies. TypeScript-first.

Perfect for rate-limited APIs, bulk database inserts, or any operation where you can't process everything at once.

Install

npm install @padmaj/batchify

Usage

Basic

import { batchify } from '@padmaj/batchify'

const results = await batchify(userIds, {
  batchSize: 10,
  handler: (batch) => fetchUsers(batch),
})

With concurrency

const results = await batchify(userIds, {
  batchSize: 10,
  concurrency: 3,  // run 3 batches in parallel
  handler: (batch) => fetchUsers(batch),
})

Log progress with onBatch

const results = await batchify(records, {
  batchSize: 50,
  concurrency: 2,
  handler: (batch) => insertToDb(batch),
  onBatch: (batch, index) => {
    console.log(`Processing batch ${index + 1}...`)
  },
})

API

batchify(items, options)

| Option | Type | Default | Description | |---|---|---|---| | batchSize | number | 10 | Number of items per batch | | concurrency | number | 1 | Max batches processed in parallel | | handler | (batch: T[]) => Promise<R[]> | required | Function to process each batch | | onBatch | (batch: T[], index: number) => void | — | Called before each batch is processed |

Returns Promise<R[]> — all results flattened into a single array, in the same order as the input.

Notes

• Results are always returned in the same order as the input, even with concurrency > 1.
• If handler throws, the error propagates immediately and remaining batches are not processed.
• batchSize and concurrency must be at least 1, otherwise a RangeError is thrown.

License

MIT