Concurrent Map

A concurrent version of Array.map that limits the number of concurrent asynchronous operations.

This package allows you to process large arrays of items with an asynchronous function (asyncFn) while controlling how many promises run in parallel. This is useful to prevent resource exhaustion (e.g., API rate limits, database connections) when dealing with many asynchronous tasks.

Installation

Install via npm:

npm install @silyze/concurrent-map

Import

In CommonJS:

const concurrentMap = require("@silyze/concurrent-map");

In ES Modules / TypeScript:

import concurrentMap from "@silyze/concurrent-map";

API Reference

export default async function concurrentMap<T, R>(
  items: T[],
  asyncFn: (item: T) => Promise<R>,
  maxConcurrency: number = 8
): Promise<R[]>;

Parameters

• items: T[] The array of input items to be processed.

• asyncFn: (item: T) => Promise<R> An asynchronous function that takes an item of type T and returns a Promise resolving to a result of type R. This function will be invoked for each element in items.

• maxConcurrency: number (optional, default: 8) Maximum number of asyncFn calls to run in parallel. Once this limit is reached, concurrentMap waits for the first pending promise to settle before scheduling a new one.

Returns

• Promise<R[]> A promise that resolves when all asyncFn invocations have settled (either fulfilled or rejected). The resolved value is an array of results of type R. Note that the order of results in the array corresponds to completion order, not input order. If you need to preserve original order, see the "Ordering Results" section.

Usage Examples

Basic Example

import concurrentMap from "@silyze/concurrent-map";

(async () => {
  const items = ["a", "b", "c"];

  const results = await concurrentMap(
    items,
    async (item) => {
      // Simulate an async task
      await new Promise((resolve) => setTimeout(resolve, 1000));
      console.log(`Processed: ${item}`);
      return item.toUpperCase();
    },
    2
  );

  console.log(results); // e.g. ['A', 'B', 'C'] (order may vary)
})();

In this example:

1. Two tasks run concurrently (maxConcurrency = 2).
2. Each item is processed with a 1-second delay.
3. The final results array contains the uppercase letters, but may not be in the original input order.

Preserving Input Order

If you need the output array to maintain the same order as the input array, wrap each result with its index and sort after processing:

import concurrentMap from "@silyze/concurrent-map";

(async () => {
  const items = ["a", "b", "c"];

  const indexedResults = await concurrentMap(
    items.map((value, index) => ({ index, value })),
    async ({ index, value }) => {
      const result = await someAsyncOperation(value);
      return { index, result };
    },
    3
  );

  // Sort by original index
  indexedResults.sort((a, b) => a.index - b.index);

  // Extract values in order
  const orderedResults = indexedResults.map((item) => item.result);
  console.log(orderedResults);
})();

Error Handling

If any invocation of asyncFn rejects, concurrentMap still waits for all pending tasks to settle before resolving the returned promise. Rejected tasks will be omitted from the results array; if you need to capture errors, you can catch them inside asyncFn and return an object with either value or error:

import concurrentMap from "@silyze/concurrent-map";

type Outcome<R> =
  | { status: "fulfilled"; value: R }
  | { status: "rejected"; error: any };

(async () => {
  const urls = ["url1", "url2", "url3"];

  const outcomes = await concurrentMap(
    urls,
    async (url) => {
      try {
        const response = await fetch(url);
        const data = await response.json();
        return { status: "fulfilled", value: data } as Outcome<unknown>;
      } catch (error) {
        return { status: "rejected", error } as Outcome<unknown>;
      }
    },
    5
  );

  outcomes.forEach((outcome) => {
    if (outcome.status === "fulfilled") {
      console.log("Data:", outcome.value);
    } else {
      console.error("Error:", outcome.error);
    }
  });
})();

Performance Considerations

• Batch Size: Choose maxConcurrency based on your environment (CPU, memory, network). Too high can exhaust resources; too low underutilizes.
• Task Duration: If tasks vary significantly in duration, the order of completion (and thus result positions) will reflect that.