resilix

Load limiting for JavaScript.

A dependency that has slowed to a crawl is still "up". resilix measures latency, works out how much concurrency your upstream can actually absorb, and sheds the rest — before the errors start.

Zero dependencies. No I/O.

Documentation: resilix.js.org

Runs on Node 18/20/22/24, Bun, Deno and Cloudflare Workers — verified in CI on every push by importing the built artifact and driving a real breaker, not asserted. The Workers job specifically proves a module-scope import is side-effect free, which is the thing that crashes some libraries at the edge.

npm i resilix

Why this exists

JavaScript has fault handling. Its load limiting is locked inside two RPC clients — hedging and retry budgets only in @grpc/grpc-js, adaptive throttling only in the AWS SDK, ~90M downloads a week between them, and unavailable to anyone calling a plain HTTP API, a database or a queue.

Two failure modes motivated this library, both from production:

1. Your upstream degrades without erroring. A provider went from p50 0.35s / p95 0.9s to p50 10.4s / p95 15.3s — roughly 25–30× slower with a completely flat error rate. A failure-rate circuit breaker is blind to that until calls start timing out. resilix trips on slow-call rate, a dimension no other JavaScript breaker has.

2. Healthy traffic contains a lot of 4xx. On a good day, 13–18% of calls to a validating upstream returned 4xx. Any breaker whose failure predicate is "did the promise reject?" opens because customers submitted bad input. resilix classifies outcomes into verdicts, so a 4xx is answered — healthy — while a 429 is overload: not a failure, but still a load signal.

See it work

pnpm example:gateway

A simulated provider degrades from ~140ms to seconds at a flat error rate, then recovers. Watch the limiter walk concurrency down before any failures appear — examples/llm-gateway, written up at resilix.js.org/guide/example.

Quick start

import { pipeline, breaker, classifyHttp } from "resilix";

const api = pipeline<Request>({
  key: (req) => new URL(req.url).host,   // one breaker per host
  classify: classifyHttp,
  timeoutMs: 15_000,
  policies: [
    breaker({
      slowCallMs: 3_000,        // required: ~3x your healthy p95
      slowCallRate: 0.5,        // trip if >50% of calls are slow
      failureRate: 0.5,         // ...or if >50% fail
      consecutiveBackstop: 10,  // ...or 10 in a row, at any traffic level
    }),
  ],
});

const res = await api.execute(req, (ctx) => fetch(req.url, { signal: ctx.signal }));

Refused calls throw RejectedError with a reason and, where known, retryAfterMs.

Streaming: mark the latency that matters

A call's latency defaults to its total duration. That is wrong for anything streaming — a 45-second LLM completion is healthy if the first token arrived in 300 ms. Judge it on total duration and every healthy stream looks slow, so the slow-call breaker opens on a perfectly good upstream. Call ctx.mark() at the moment that actually indicates health:

await api.execute(req, async (ctx) => {
  const res = await fetch(url, { signal: ctx.signal });
  ctx.mark();                 // time to first token — the health signal
  return consumeStream(res);  // may run for another 45s; not counted
});

Watch for a starved window

A window bounded by age holds at most maxAgeMs / callDuration samples, so an upstream whose calls take longer than maxAgeMs / minCalls — 15 s at the defaults — can never reach minCalls. That used to leave both rate conditions permanently inert.

Two things prevent it now. The age bound is widened automatically to minCalls × slowCallMs when the configured one is too narrow to hold that many samples — read breaker.stats().effectiveMaxAgeMs for the value in force. And once the age bound is actually evicting, the breaker will decide on what it has (down to 5 samples) rather than waiting for minCalls that can never arrive: every sample that exists is already in the window.

breaker.stats().starved and the resilix.breaker.starved gauge still report when the window is running below minCalls, which is worth alerting on as a sign your slowCallMs and window bounds do not match your workload.

Guarding a database

classifyHttp is wrong for SQL: it calls a unique-violation transient, so a burst of duplicate inserts looks like the database falling over. Use classifySql:

import { classifySql } from "resilix";
pipeline({ classify: classifySql, policies: [bulkhead({ concurrency: 10 }), breaker({ ... })] });

Every mapping was verified against real errors from pg 8.23 and Prisma 7.9 on PostgreSQL 16, not from documentation. Three things only showed up that way:

• pg pool exhaustion has no error code at all — a bare Error reading "timeout exceeded when trying to connect". Misread as transient, a burst that exhausts your pool would open the circuit, when the database is healthy and you should shed load.
• Prisma 7 nests the real SQLSTATE at meta.driverAdapterError.cause.originalCode, and its P2010 is ambiguous — the same code wraps a syntax error, a missing column and a statement timeout. classifySql unwraps it; classifying on P2010 alone calls a timeout transient.
• PrismaClientValidationError carries no code, only a name. It is the caller passing the wrong type, so it is answered and must never open a circuit.

Run pnpm test:integration with a Postgres to re-verify after a driver upgrade; the captured fixtures alone would keep passing if a shape changed.

The verdict model

One settled call, read differently by each policy. This table is the design:

| Outcome | Verdict | Breaker | Retry | |---|---|---|---| | 200 fast | success | healthy | done | | 200 in 9 s | success | counts toward slow-rate | done | | 404, 422 | answered | healthy | never | | 429, 503 | overload | healthy | after Retry-After | | 500, ECONNRESET, unlabelled | transient | failure | yes | | our deadline elapsed | timeout | failure | once | | we refused it | rejected | ignored | no |

That last row matters more than it looks. Our own shedding must never be recorded as evidence about the upstream — without it, an open breaker observes its own rejections and can never close.

Adaptive concurrency limiting

The part nothing else in npm has, and the reason this library exists.

import { pipeline, breaker, limiter } from "resilix";

const api = pipeline({
  key: (req) => req.model,
  policies: [
    breaker({ slowCallMs: 3_000 }),
    limiter(),              // infers the right concurrency from latency
  ],
});

A circuit breaker is binary: open or closed. A limiter is continuous — it works out how many concurrent calls your upstream can actually absorb, from latency alone, and sheds the excess. Latency rises before errors do, which is why this catches degradation a failure-rate breaker cannot see at all.

| | | |---|---| | Algorithm | Vegas queue estimation by default (gradient2, aimd also available) | | Signal | p90 of recent latency, via an O(1) P² estimator | | Over the limit | queue to 2×, then shed proportionally to 3× — not a cliff | | 429 / timeout | short-circuits the control loop; no waiting for the next interval | | 4xx | a latency sample, but no pressure — the upstream did real work |

If you stream, call ctx.mark(). The limiter judges on time-to-first-token; feed it total duration and a healthy 45-second completion looks like saturation.

Two behaviours worth knowing. Growth is tethered to observed concurrency, so a limit of 200 is never invented while ten calls are in flight — the tether caps growth only and never shrinks the limit during a lull. And because the control loop runs on call settlement rather than a timer (resilix has no timers), staleAfterMs exists to stop a limiter clamped during an incident from staying clamped forever once traffic goes quiet.

Retry, budgets and throttling

import { pipeline, breaker, limiter, throttler, budget } from "resilix";

const shared = budget({ ratio: 0.1 });   // ONE instance for the whole process

const api = pipeline({
  policies: [throttler(), breaker({ slowCallMs: 3_000 }), limiter()],
  retry: { maxAttempts: 3, jitter: "full", budget: shared },
  timeoutMs: 10_000,
});

Retries are an amplifier. Three attempts per request turns a degraded upstream into a 3× load spike exactly when it can least absorb one. A 10% budget holds that to ~1.1× — Google SRE's number, and one this repo reproduces in a test rather than quoting.

The budget is a shared object. A per-pipeline cap cannot bound system-wide amplification, which is the entire point of having one.

Which failures are retried falls out of the verdict model: answered never (the upstream worked, the caller was wrong), rejected never (we refused it), transient / timeout / overload yes — and overload waits for the upstream's own Retry-After in preference to any backoff curve.

timeoutMs bounds the whole sequence, not each attempt. Most libraries bound each attempt, so a caller asking for 50 ms can wait maxAttempts × (50 ms + backoff). A deadline the caller cannot see is not a deadline.

Four ways to be refused

| Refused by | reason | Means | |---|---|---| | breaker | circuit-open | the upstream looks wholly down | | limiter | limiter-full | too many in flight for current latency | | throttler | throttled | too many recent attempts were not accepted | | bulkhead | bulkhead-full | a hard concurrency cap you configured | | rate limiter | rate-limited | a fixed rate you configured | | budget | budget-exceeded | the retry was refused; the first attempt was not |

Every one arrives as RejectedError.reason and on onRejection, so "why was I refused?" always has an answer.

Hedging, criticality and fairness

const api = pipeline<Req>({
  key: (r) => r.host,
  priority: (r) => (r.background ? "bulk" : "critical"),
  tenant: (r) => r.orgId,
  policies: [throttler(), breaker({ slowCallMs: 3_000 }), limiter()],
  hedge: { idempotent: true },        // delay defaults to the measured p95
  retry: { budget: shared },
});

Hedging races a second attempt against a slow first one and cancels the loser. The delay defaults to the measured p95 for that key, not a constant — Dean & Barroso's ~2% overhead is a consequence of hedging at a high percentile, so a fixed number loses the property that made it cheap. It takes the first success, not the first result: a hedge that fails fast must not beat an original that would have succeeded.

idempotent: true is required, not advisory. A hedge sends the same request twice; on a payment that is a double charge.

Criticality sheds low-value work first, using Netflix's four buckets — critical, degraded, bestEffort, bulk. Unlabelled work defaults to critical, because the alternative silently sheds things nobody classified. In Netflix's own incident a 12× prefetch spike saw over half of all requests throttled while user-initiated availability stayed above 99.4% — the load landed entirely on work nobody was waiting for.

Fairness is relative, not quota-based: under pressure the tenant furthest above admitted / activeTenants is shed first, and heaviness decays so nobody is punished forever. No number to configure and nothing to keep up to date.

When a circuit breaker is the wrong tool

Worth saying plainly, because it is the best-known criticism of the pattern and it is correct. Marc Brooker's argument, in short: circuit breakers turn partial failures into complete ones. If one shard of a sharded backend is overloaded while the rest are healthy, a breaker either trips — degrading every caller hitting the healthy shards — or it does not trip, in which case it is doing nothing. His example is heterogeneous load: one key range gets hammered, the others idle, and the client cannot tell from outside whether the backend is down or merely hot for particular parameters.

Three practical consequences:

• Key by the thing that fails independently, not by host. key: (req) => req.shardId or key: (req) => req.tenant is usually more correct than keying by hostname. resilix keys by whatever you return, so this is your choice to get right.
• If the failure domain is not visible in the request, do not use a breaker. No key choice helps when you cannot see which shard you are talking to. Shed load proportionally instead — that is what the adaptive limiter is for (v0.3), and a partial outage then produces partial shedding rather than an all-or-nothing decision.
• A breaker is right for a homogeneous upstream that is wholly up or wholly down. That is the case resilix was built for: one provider, one endpoint, degrading as a unit.

Three trip conditions

1. failure rate  > threshold, over a dual-bound window   (n >= minCalls)
2. SLOW-CALL rate > threshold, over the same window      (n >= minCalls)
3. consecutive failures >= backstop                     (window-independent)

The window is dual-bound — the last calls samples and only those within maxAgeMs. Count-only windows go stale at low traffic (at 8 req/min, "the last 100 calls" spans ~12 minutes). Time-only windows are unbounded at high traffic.

Condition 3 closes a hole every rate-based breaker has. Rate conditions cannot fire below minCalls, so a completely dead upstream at low traffic never accrues enough samples and therefore never trips — every caller eats the full timeout.

Half-open admits exactly one probe by default, so recovery cannot stampede an upstream that is by definition fragile. It self-heals if a probe is admitted and never settles.

You can own the call

execute() is convenience. Every policy is a synchronous state machine, so you can drive it directly — from a stream consumer, a queue worker, or anywhere a promise wrapper is in the way:

const gate = api.gate(req);
if (!gate.ok) throw new Error(`refused: ${gate.reason}`);

const started = performance.now();
try {
  const res = await fetch(req.url);
  gate.settle(res, performance.now() - started);
} catch (err) {
  gate.settle(err, performance.now() - started);
}

Design commitments

• Zero runtime dependencies, no I/O in core. A policy decision costs microseconds and cannot itself fail.
• Time is injected. Every temporal behaviour is deterministically testable; the test suite has no real timers and no sleep().
• Nothing at module scope. No timers, no AbortController, no random values at import time — which is what makes some libraries crash wrangler dev on import.
• O(1) per call. The window keeps running counters, so rates never iterate and eviction is a tail advance, not a scan. Typed arrays, preallocated, no steady-state allocation.
• Bounded key registry. Per-host state has a TTL and a hard cap, so tenant- or attacker-influenced keys cannot leak memory.
• No distributed state. Cross-instance ejection belongs to the service mesh. snapshot() / hydrate() cover the serverless case, which is what people actually need — and they are origin-safe: every serialised time is relative, and idle time between processes is accounted for, so a rehydrated window ages correctly instead of coming back looking fresh.

Status

Pre-release.

• v0.1 classifier · circuit breaker · dual-bound window · key registry · pipeline executor
• v0.2 resilix/otel · resilix/compat/opossum · bulkhead · observers
• v0.3 adaptive concurrency limiting · P² streaming quantiles · proportional shedding — built to docs/specs/adaptive-limiter.md
• v0.4 retry with full jitter · shared retry budgets · SRE adaptive throttler · token-bucket rate limiter
• v0.5 hedging with cancellation · criticality buckets · tenant fairness
• v0.6 resilix/undici — guard the dispatcher, and every call site in the process is covered

What is still ahead — resilix/nest and resilix/hono, then inbound protection: shedding load you receive rather than load you send — is in docs/public/resilix-architecture.pdf, along with the C4 architecture and the reasoning behind every default.

Guarding undici (resilix/undici)

resilix/fetch covers the WHATWG API, but Node services reach the network through undici's Dispatcher — which is what fetch, undici.request and most SDK HTTP layers sit on. Guarding the dispatcher covers every call site at once, including ones you do not own:

import { Agent, setGlobalDispatcher } from "undici";
import { resilixInterceptor } from "resilix/undici";

setGlobalDispatcher(
  new Agent().compose(
    resilixInterceptor({ policies: [breaker({ slowCallMs: 3_000 }), limiter()] }),
  ),
);

Latency is time to first byte, taken at onResponseStart — never the time to drain the body. A refused request never reaches the network; the RejectedError arrives through onResponseError so it surfaces like any other undici failure.

Requires undici >= 7 — undici 7 renamed the whole handler surface, and compose() exists on 6.x too, so an undici 6 install would get a silently inert interceptor. undici is an optional peer; core stays at zero dependencies.

timeoutMs, retry and hedge are not available through this adapter. dispatch is callback-driven and returns synchronously, so the executor cannot wrap it — compose undici's own interceptors.retry and headersTimeout alongside instead. Every policy works unchanged.

Telemetry (resilix/otel)

Built in, not a plugin. Under 1% of opossum users instrument their breakers, which means almost nobody has data at the moment they need it.

import { metrics } from "@opentelemetry/api";
import { otel } from "resilix/otel";

const instrument = otel({ meter: metrics.getMeter("checkout") });
const api = pipeline({ policies: [...], observers: [instrument] });
instrument.observeGauges(api);   // pull-based gauges

| Instrument | Type | Attributes | |---|---|---| | resilix.executions | counter | key, verdict | | resilix.execution.duration | histogram (ms) | key, verdict | | resilix.rejections | counter | key, reason, policy | | resilix.state.transitions | counter | key, from, to, reason | | resilix.breaker.{state,failureRate,slowRate,windowSize} | gauge | key | | resilix.bulkhead.{inFlight,limit,utilisation} | gauge | key |

@opentelemetry/api is an optional peer dependency — core stays at zero deps. Without a meter, otel() is a no-op, so tests need no OTel install. Observers are dispatched through a swallowing wrapper: a failing exporter can neither influence nor break an admission decision.

Migrating from opossum (resilix/compat/opossum)

Scope of the claim, measured: the shim passes 362 of 362 of opossum's own test suite, run unmodified against resilix/compat/opossum. Reproduce it yourself with pnpm test:compat — it fetches their suite, points their require('../') at our build, and fails if this README's number is out of date.

Three of their test files are excluded because no compatibility layer can ever satisfy them: cache.js, semaphore-test.js and status-test.js require('../lib/…') directly, so they unit-test opossum's private modules rather than its public API. Caching and call coalescing are also unimplemented on purpose — passing cache, coalesce or cacheTTL throws rather than silently doing nothing.

- const CircuitBreaker = require('opossum');
+ const CircuitBreaker = require('resilix/compat/opossum');

Default behaviour is opossum's, not resilix's — a compat layer must not change what your service does on the day you swap the import. slowCallRate defaults to 1 (disabled) and consecutiveBackstop to 0 (disabled), because opossum has neither concept.

| opossum option | Mapping | |---|---| | timeout | pipeline deadline; false disables | | errorThresholdPercentage | failureRate (÷100) | | resetTimeout | openForMs | | rollingCountTimeout | window.maxAgeMs | | rollingCountBuckets | accepted and ignored — our window is not bucketed | | volumeThreshold | window.minCalls | | errorFilter | wrapped into a classifier: true ⇒ not a failure | | capacity | bulkhead({ concurrency }) | | cache, coalesce, cacheTTL | throws. Out of scope — silently accepting them would be worse |

Supported: fire, fallback, on/off/removeAllListeners, open/close, enable/disable, opened/closed/halfOpen/pendingClose, stats, status, isOurError, and the fire/success/failure/timeout/reject/open/close/halfOpen/ fallback/semaphoreLocked events.

Opt back into the resilix behaviour when you're ready:

new CircuitBreaker(action, { slowCallMs: 3000, slowCallRate: 0.5, consecutiveBackstop: 10 });

Documentation

The full docs are at resilix.js.org — the same content as this README, organised into a guide, plus the decisions and specs below.

| | | |---|---| | docs/decisions.md | why resilix is shaped the way it is — every ADR-00N comment in the source resolves here, at /decisions#adr-007 | | docs/specs/ | the design specs, written before the code and carrying every default's provenance | | CONTRIBUTING.md | the one rule that gets broken most, and how the build now enforces it | | READING.md | the sources behind each version, and what remains unread |

License

MIT © Musa Musa