@idle-runner/worker

Send heavy work to a worker and await the result. ~1.4 kB gzipped on the main thread, ~1.35 kB in the worker, zero dependencies.

@idle-runner/core defers work on the main thread — a 200ms computation is still 200ms of main thread, just chopped up. This package moves it off the main thread entirely: a named task registry inside the worker, one promise per call on the outside, plus the two things hand-rolled postMessage code always ends up missing — cancellation and transferables.

There is no dependency between the two packages. Use whichever the work calls for.

Install

npm install @idle-runner/worker

Quick start

The worker declares what it can do:

// worker.ts
import { defineWorkerTasks } from '@idle-runner/worker';

export const tasks = {
    buildIndex: (products: Product[]) => buildSearchIndex(products), // heavy and synchronous
    parse: (csv: string) => parseCsv(csv),
};

defineWorkerTasks(tasks);

The main thread calls it by name:

// app.ts
import { createWorkerRunner } from '@idle-runner/worker';
import type { tasks } from './worker';

const runner = createWorkerRunner<typeof tasks>(
    () => new Worker(new URL('./worker.ts', import.meta.url), { type: 'module' })
);

const index = await runner.push('buildIndex', products);

import type is erased at compile time, so importing the worker module for its types puts nothing worker-side in your page bundle — but it gives push the real payload and result types, per task name.

The worker itself is created on the first push, not when the runner is made. A runner nobody uses costs nothing, and a module that is never called is safe to import on a server.

Chunked work: yield in the worker

A worker has no rendering to block, so a task starts the moment the worker is free — there is no idle budget and nothing to wait for. yield still has a job, though: it is where a cancellation can land.

defineWorkerTasks({
    downscale: function* (photos: Photo[]) {
        const thumbs: Thumbnail[] = [];

        for (const photo of photos) {
            thumbs.push(downscale(photo));
            yield; // an abort sent from the page can take effect here
        }

        return thumbs;
    },
});

The task resolves with the generator's return value; what it yields is never a result. Steps run back-to-back and control goes back to the worker's event loop about every 16ms — often enough that an abort is picked up promptly, rarely enough that a generator yielding 100k times does not become 100k macrotasks.

A task may also be async; the promise is awaited and the call resolves with its value.

Cancelling

const controller = new AbortController();
const thumbs = runner.push('downscale', photos, { signal: controller.signal });

controller.abort(); // `thumbs` rejects with AbortError right now

The rejection does not wait for a round trip. The caller is answered immediately and the worker is told in parallel, because a worker in the middle of a synchronous task is not reading messages at all — waiting for it to acknowledge would mean waiting for the very work you are trying to cancel.

What the worker does with the abort depends on the task:

| the call is… | effect in the worker | | -------------------------------- | ------------------------------------------------------------------- | | still queued behind another task | dropped; never starts | | a generator, mid-flight | stops at its next yield, and its finally blocks run | | a plain function, mid-flight | runs to the end — nothing can interrupt it; the result is discarded | | an async task, mid-flight | same: the promise settles, the result is discarded |

That last row is not a limitation of this library but of the platform: a synchronous function owns its thread until it returns. Chunk it with yield if you need it to be cancellable.

Transferables

Binary payloads should be handed over, not copied — in both directions.

// main thread: the buffer is detached here and adopted there, with no copy
const stats = await runner.push('analyse', buffer, { transfer: [buffer] });

// worker: return by transfer instead of by copy
import { defineWorkerTasks, transfer } from '@idle-runner/worker';

defineWorkerTasks({
    decode: (bytes: ArrayBuffer) => {
        const pixels = heavyDecode(bytes);

        return transfer(pixels, [pixels.buffer]);
    },
});

transfer() is typed as the value it wraps, so the task's return type stays the real result type and push resolves with exactly that. The consequence: return it straight from the task — do not read the wrapper or reuse it.

Reading a transferred object afterwards throws, on either side. That is the point: there is only one owner.

Errors

A task that throws rejects its call, and nothing else — neighbouring calls are unaffected and the worker keeps serving:

try {
    await runner.push('parse', csv);
} catch (error) {
    // error.name === 'SyntaxError', error.message and error.stack all survive,
    // and the stack points into the worker script, where the throw happened.
}

Errors are taken apart and rebuilt across the wire rather than structured-cloned. Cloning an Error drops its subclass and throws outright if the instance carries a non-cloneable property — losing the failure entirely, which is the one thing an error channel may not do. Name, message and stack survive; a thrown non-Error value is passed through as-is.

Some failures belong to the worker rather than to a task. Each one rejects everything in flight rather than leaving promises pending forever, because none of them can be attributed to a single call:

• a worker that never started — an uncaught error before it has answered anything, most often a script that failed to load;
• a reply that could not be deserialised;
• a payload the worker could not deserialise, which never reaches a task and so can never be answered by id.

Only the first is fatal. A worker that has never spoken is assumed to be dead — a push would post into a void and wait for a reply that cannot come — so the runner is marked as crashed and later pushes reject too, with the original failure and a note about how to recover. The other two say nothing about the health of the thread, so the runner keeps using it.

An uncaught error in a worker that has already answered a call is a different thing: the worker is demonstrably alive, and the throw belongs to something else running in it — a stray timer, an unrelated listener, a library with an opinion. Workers survive uncaught errors, so calls in flight still get their answers and the runner keeps serving. The error is reported through onError if you passed one; the browser logs it either way. A node:worker_threads thread does not survive one — see Node.js.

Recovery from a crash is manual by default: terminate(), then push again to get a fresh worker. Pass autoRestart to have the runner do it for you — the dead worker is terminated and unhooked, and the next push starts a new one:

const runner = createWorkerRunner(factory, { autoRestart: true });

Calls in flight are still rejected, and nothing is retried: a payload that killed a worker would kill its replacement just as reliably, so re-sending it is how you turn one crash into a loop. Retry at the call site, where you know whether the call is safe to repeat. For the same reason the restarts are budgeted — after maxRestarts (default 3) consecutive workers die before answering, the runner stops respawning and marks itself crashed, which is what a script that cannot load looks like. A worker that has answered at least once is never restarted: it is demonstrably alive, and its uncaught errors belong to something else running in it.

Pooling

One runner owns one worker, and one worker runs one call at a time — a second push waits for the first to finish. When the work is genuinely parallel, createWorkerPool spreads it over several workers behind the same interface:

import { createWorkerPool } from '@idle-runner/worker';

const runner = createWorkerPool<typeof tasks>(
    () => new Worker(new URL('./tasks.worker.ts', import.meta.url), { type: 'module' }),
    { size: 4 }
);

const [a, b, c] = await Promise.all([
    runner.push('resize', first),
    runner.push('resize', second),
    runner.push('resize', third),
]); // three threads, not three turns

push, clear, terminate and size mean exactly what they do on a single runner, so swapping one in is a one-line change. Each call goes to the least busy worker, and workers are still created lazily — a pool that never sees two calls at once never starts a second thread. size defaults to hardwareConcurrency - 1, capped at 4, and to 2 where the browser does not report hardwareConcurrency.

Workers share nothing. A task that caches something between calls, or that expects to see every call, must stay on a single createWorkerRunner — on a pool it would see an arbitrary subset.

For calls nobody awaited — and for a non-fatal error inside the worker, which belongs to no call at all — pass onError when creating the runner. Aborts are never reported through it.

When to use this — and when not to

Good fits — CPU-bound work with a serialisable payload and a serialisable result:

• parsing and transforming large payloads (CSV, JSON, protobuf)
• building search indexes, diffs, layouts, aggregations
• image, audio and video processing on ArrayBuffers
• crypto, compression, WASM number-crunching

Bad fits — use something else:

• ❌ Work that touches the DOM. There is no document in a worker. Compute in the worker, apply on the main thread.
• ❌ Work whose payload dwarfs the work. Structured clone is not free; shipping 50MB to save 2ms of computation is a net loss. Transferables help when the data is binary and you no longer need it locally.
• ❌ Non-urgent work that must stay on the main thread (touching DOM, waiting on layout) — that is @idle-runner/core's job.
• ❌ Async job concurrency control (rate-limiting N fetches) — that's p-queue's job.

Does it actually work?

Tested in real browsers rather than asserted, in test/browser/worker.browser.test.ts — real Worker, real structured clone, real transfers — with a negative control, because a benchmark that only shows the good number proves nothing:

| workload | long tasks (≥50ms) on the main thread | | -------------------------------------------------- | ------------------------------------- | | 180ms of work run on the main thread (the control) | at least one — as it must | | the same 180ms of work inside the worker | none |

A companion test keeps a requestAnimationFrame loop running while the worker chews through 150ms of work and asserts that frames keep arriving. The transfer tests assert the buffer is detached on the sending side in both directions — the only observable proof that nothing was copied.

Everything runs on Chromium and WebKit in CI. The two long-task assertions are skipped on WebKit, which does not implement the longtask entry type; every other test runs on both.

How it works

One worker per runner, one message per call, and nothing clever:

| direction | message | meaning | | --------- | -------------------------- | ---------------------------- | | → worker | { id, name, payload } | call this task | | → worker | { id, op: 'abort' } | drop this call if you can | | → page | { id, ok: true, value } | it resolved | | → page | { id, ok: false, error } | it threw | | → page | { op: 'undeliverable' } | a payload arrived unreadable |

Each call gets a numeric id and an entry in a Map<id, {resolve, reject}>; replies are matched by id, and a reply for an id that is no longer there — cancelled, cleared — is dropped. A message needs an ok to count as a reply at all, so a worker script that posts messages of its own alongside the task protocol cannot settle somebody's call by reusing a number.

Inside the worker, calls run one at a time through a FIFO queue. That queue is a correctness requirement rather than a policy choice: a generator task hands the event loop back between slices, so without it the message for a second call would start executing nested inside the first, interleaving two tasks that each believe they own the thread.

Both halves ship from one entry point. They share nothing at runtime and the package is side-effect-free, so a bundler drops createWorkerRunner from your worker chunk and the worker half from your page chunk.

SSR / Node

Safe to import on a server. No worker is created until the first push, and defineWorkerTasks() outside a worker realm warns and no-ops instead of installing a listener that could never fire. No typeof window guard, no dynamic import. Covered by test/ssr.test.ts.

A worker realm here means the Web Worker one — a global postMessage and addEventListener, and no window. Node's worker_threads is not that: there the channel is parentPort, so defineWorkerTasks() from the main entry warns and registers nothing. Node threads have their own entry, @idle-runner/worker/node, which speaks the same protocol over parentPort.

If the environment has no Worker at all, the failure surfaces as a rejected push — never as a throw out of createWorkerRunner or at import time.

Node.js (worker_threads)

@idle-runner/worker/node is the same package against node:worker_threads: same protocol, same push(name, payload), same cancellation and transferables. Only the two ends change — a factory that returns a Node Worker, and a task host that listens on parentPort.

// tasks.worker.ts
import { defineWorkerTasks, transfer } from '@idle-runner/worker/node';

export const tasks = {
    parse: (csv: string) => parseCsv(csv),
    hash: (buffer: ArrayBuffer) => transfer(sha256(buffer), [buffer]),
};

defineWorkerTasks(tasks);

// app.ts
import { Worker } from 'node:worker_threads';
import { createWorkerRunner } from '@idle-runner/worker/node';
import type { tasks } from './tasks.worker.js';

const runner = createWorkerRunner<typeof tasks>(
    () => new Worker(new URL('./tasks.worker.js', import.meta.url))
);

const rows = await runner.push('parse', csv);

The URL points at the compiled worker file — Node loads a worker itself, without your bundler or a TypeScript loader in the way. import type is erased, so importing the task module for its types costs nothing at runtime.

The browser entry stays free of Node builtins — node:worker_threads and node:os are imported only from this one — so nothing here reaches a page bundle.

Two things differ from the browser, both because a Node thread is not a Web Worker:

• A thread can vanish without an error event. process.exit() inside the worker, an OOM kill, a native crash: in a browser that cannot happen, and in Node every call in flight would wait forever for a reply that is never coming. An exit that no error preceded is reported as a worker failure, so those calls reject like any other crash — with the exit code in the message. An exit caused by terminate() is not, because the runner detaches its listeners before terminating.
• A thread that fails is gone, however long it served. The rule above — a worker that has already answered a call survives an uncaught error — is a browser rule; a Node thread does not survive one, and it does not come back from an exit either. So both are fatal here whatever the thread answered first: the calls in flight reject, and later pushes reject until you terminate() or pass autoRestart.
• A live thread keeps the process alive. Call terminate() when the work is done, or unref() the worker inside the factory if it should never hold the process open.

createWorkerPool sizes itself from os.availableParallelism() rather than navigator.hardwareConcurrency, so it respects the CPU quota a container was given.

Covered end to end in test/node-worker.test.ts — real threads, real structured clone, real transfers, and a task that calls process.exit() to prove the hang is caught.

API

createWorkerRunner(factory, options?)

factory is called at most once per worker, on the first push. Passing a factory rather than a Worker is what makes creation lazy — and it keeps the new Worker(new URL(...)) form intact, which is the only form bundlers can statically detect.

| Option | Type | Default | Description | | ------------- | -------------------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | | onError | (error: unknown) => void | — | Error channel for calls nobody awaited, and for worker errors that belong to no call. Aborts are not. | | autoRestart | boolean | false | Replace a worker that died before answering instead of marking the runner crashed. Calls in flight still reject, and nothing is retried. See Errors. | | maxRestarts | number | 3 | Consecutive replacements allowed before the failure is treated as fatal. terminate() resets the budget. |

| Member | Description | | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | | push(name, payload?, opts?) | Call a task by name; resolves with its result. | | clear(reason?) | Reject every in-flight call (AbortError by default, or your reason) and tell the worker to drop them. The worker stays alive. | | terminate() | Reject every in-flight call and terminate the worker. A later push starts a fresh one — which is also how you recover from a crash. | | size | In-flight call count. |

Per-call options:

| Option | Type | Description | | ---------- | ---------------- | ----------------------------------------------------------------------------------------------------------------------- | | signal | AbortSignal | Abort this one call. Rejects with AbortError immediately. See Cancelling for what the worker can stop. | | transfer | Transferable[] | Hand these over instead of copying them. They are detached on this side. |

createWorkerPool(factory, options?)

Several workers behind the createWorkerRunner interface, each call routed to the least busy one. Takes every createWorkerRunner option, applied to each worker, plus:

| Option | Type | Default | Description | | ------ | -------- | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | | size | number | hardwareConcurrency - 1, max 4 | How many workers the pool may own. Each is created lazily. Clamped to >= 1. Falls back to 2 where navigator.hardwareConcurrency is unavailable. |

factory is called once per worker the pool actually starts. See Pooling for when a pool is the wrong tool.

defineWorkerTasks(tasks)

Runs inside the worker. Registers a plain object of task functions: a sync function, a generator function, or an async function. Each takes one payload — bundle several values into an object — and calling push on a task that declares more than one argument is a compile error against that call, not against the registry. Calling defineWorkerTasks more than once merges into the same registry and keeps a single message listener.

A task may declare its payload optional ((payload?: T) => …), in which case push will take it or leave it.

An unknown task name rejects the call with a clear error rather than going silent, and so does an async function* — an async generator is neither of the two shapes the host can drive, and a wrong answer is worse than a failure.

@idle-runner/worker/node

The Node entry re-exports the whole API with the two ends swapped for node:worker_threads:

| Export | Difference from the browser entry | | ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | | createWorkerRunner(factory, opts?) | factory returns a Node Worker instead of a Worker. Same options, same members. | | createWorkerPool(factory, opts?) | Same, and size defaults to os.availableParallelism() - 1, capped at 4. | | defineWorkerTasks(tasks) | Listens on parentPort. On the main thread — where parentPort is null — it warns and registers nothing, so the module stays importable for its types. | | transfer(value, transferables) | Unchanged. | | fromNodeWorker(worker) | The adapter itself: wraps a Node worker in the Worker interface, for code that would rather build the runner by hand. |

transfer(value, transferables)

Worker-side. Marks a result to be handed over rather than copied. Return it directly from the task.

License

MIT