@flux-gate/core

Rate limiting algorithms for Node.js: five of them, in plain TypeScript, with zero dependencies.

Use it to answer one question for every request, message or job: "should this caller be allowed to go ahead right now?" You give it a caller's key (a user id, an API key, an IP address) and it tells you yes or no, how many requests are left, and how long to wait if the answer is no.

npm install @flux-gate/core

import { TokenBucketLimiter } from '@flux-gate/core';

// Each caller can burst 20 requests, then gets 2 more every second.
const limiter = new TokenBucketLimiter({ capacity: 20, refillPerSecond: 2 });

const result = limiter.consume('user:42');
// { allowed: true, limit: 20, remaining: 19, resetMs: 500 }

if (!result.allowed) {
  // Tell the caller to come back later: wait result.retryAfterMs milliseconds.
}

That's the whole idea. Everything below helps you pick the right algorithm and use it well.

Contents

• What is it for?
• Quick start
• What consume() returns
• Choosing an algorithm
• The five algorithms
• Charging by cost, not by request
• Testing without waiting
• Using it in your app
• Things to know
• API reference
• The rest of FluxGate

What is it for?

If you run an API, sooner or later someone sends it more than it can take: a scraper, a retry loop with no backoff, a customer's script stuck in while (true), or one user burning through your LLM budget overnight. A rate limiter is what says "not so fast". Typical uses:

| You want to... | Use | |---|---| | Stop one noisy client from slowing everyone down | One limiter keyed by API key or IP | | Give free and paid plans different limits | One limiter per plan | | Slow down password guessing or SMS sending | A small, strict limiter (e.g. 5 attempts, refilled every 10 s) | | Meter an LLM or any usage-priced API | A limiter charged in tokens, not requests (cost) | | Cap messages per connection on a WebSocket, or jobs per tenant in a queue worker | The same limiter, called from your own code |

This package is the engine: pure, synchronous classes that hold their state in memory. It has no network code and no framework code, so it runs anywhere JavaScript runs on Node 20 or newer. When you need a limit shared by several servers, or middleware for Express, Fastify, Hono or Next.js, the sibling packages build on this one (see below).

Quick start

consume(key) is the only method you need. It checks the limit, records the request if it's allowed, and returns the decision. It is synchronous, so it works in hot paths without await.

import { TokenBucketLimiter } from '@flux-gate/core';

const limiter = new TokenBucketLimiter({ capacity: 3, refillPerSecond: 1 });

limiter.consume('user:42'); // { allowed: true,  limit: 3, remaining: 2, resetMs: 1000 }
limiter.consume('user:42'); // { allowed: true,  limit: 3, remaining: 1, resetMs: 2000 }
limiter.consume('user:42'); // { allowed: true,  limit: 3, remaining: 0, resetMs: 3000 }
limiter.consume('user:42'); // { allowed: false, limit: 3, remaining: 0, resetMs: 3000, retryAfterMs: 1000 }

limiter.consume('user:7');  // a different key has its own, full bucket

A typical guard in a plain HTTP handler:

const result = limiter.consume(apiKey);

res.setHeader('X-RateLimit-Limit', result.limit);
res.setHeader('X-RateLimit-Remaining', result.remaining);

if (!result.allowed) {
  res.setHeader('Retry-After', Math.ceil((result.retryAfterMs ?? 0) / 1000));
  res.statusCode = 429; // Too Many Requests
  res.end();
  return;
}

You rarely have to write that by hand: @flux-gate/http does it, with the correct status codes and headers, in one line of middleware.

What consume() returns

Every algorithm returns the same shape, so you can switch algorithms without touching the code that reads the result.

| Field | Type | Meaning | |---|---|---| | allowed | boolean | true if the request may go ahead. | | limit | number | The configured maximum: the requests per window, or the bucket's capacity. | | remaining | number | How many more units would be allowed right now. Never negative. | | resetMs | number | Milliseconds until this key is back to a completely fresh state. | | retryAfterMs | number or absent | Only on a denial: milliseconds until a retry can succeed. If a denial has no retryAfterMs, waiting can never help, because the request's cost is bigger than the whole limit. Treat that as "too large", not "try later". |

Only allowed requests are recorded. A denied request costs the caller nothing, so a client that keeps hammering a closed door doesn't push its own reopening further away.

Choosing an algorithm

If you're unsure, use TokenBucketLimiter. It's the most widely used, it lets you tune burst size and sustained rate separately, and it's the one that has a Redis version for multi-server setups.

| Algorithm | State per key | Allows bursts? | Exactness | Pick it when | |---|---|---|---|---| | TokenBucketLimiter | 2 numbers | Yes, up to capacity | Exact for its own rule | Default. You want "burst of N, then R per second". | | GcraLimiter | 1 number | Yes, up to burst | Same decisions as the token bucket | You want the token bucket with the smallest possible state. | | SlidingWindowLogLimiter | One timestamp per allowed request | No | Exact: never more than limit in any window | Small limits where you need the strict guarantee "at most N in any windowMs". | | SlidingWindowCounterLimiter | 2 counters | No sharp edge | Approximate | You want "N per window" cheaply for many keys and can tolerate small errors. | | FixedWindowLimiter | 1 counter | Edge burst | Weakest | You only need a rough ceiling and want the simplest thing. |

What "exactness" means in numbers, from the benchmarks in the FluxGate repository (limit 100 per 60 s, 2 simulated hours of traffic per pattern; 1.00 means the limit was never exceeded):

| Peak admitted in any 60 s window | Poisson, 0.9x the limit | Bursty (10x spikes) | Adversarial (full limit on both sides of every window edge) | |---|---|---|---| | Sliding window log | 1.00x | 1.00x | 1.00x | | Sliding window counter | 1.13x | 1.59x | 1.01x | | Fixed window | 1.23x | 2.00x | 2.00x | | Token bucket / GCRA | 1.23x | 1.99x | 1.01x |

The fixed window's flaw is simple: a caller who knows where the window edges are can use the whole limit just before an edge and the whole limit again just after it, getting double the limit in a moment. The token bucket and GCRA also admit up to twice the limit in one window, but that is by design: a bucket refills continuously, and the burst is exactly the capacity you chose. You are choosing a rule, not hitting a flaw.

The five algorithms

Every limiter is built from a small config object and validates it, throwing a RangeError for a zero or negative number rather than silently allowing or blocking everything later.

Token bucket

Each key has a bucket that holds up to capacity tokens. Each request takes a token. Tokens drip back at refillPerSecond. Nothing runs on a timer: the refill is computed from the elapsed time on each request, so idle keys cost nothing to maintain.

import { TokenBucketLimiter } from '@flux-gate/core';

const limiter = new TokenBucketLimiter({
  capacity: 20,        // the biggest burst a caller can make
  refillPerSecond: 2,  // the long-run average rate
});

Used by Stripe's API limiter, AWS API Gateway throttling and Envoy's local rate limit.

GCRA (Generic Cell Rate Algorithm)

The same decisions as a token bucket, but it stores a single timestamp per key instead of two numbers (the tests verify this on random traffic). Configured by burst (how many back-to-back requests are allowed) and ratePerSecond.

import { GcraLimiter } from '@flux-gate/core';

const limiter = new GcraLimiter({ burst: 20, ratePerSecond: 2 });

Sliding window log

Remembers the time of every allowed request in the last windowMs and counts them. The window moves with each request, so there is no edge to exploit. The price is memory that grows with the limit.

import { SlidingWindowLogLimiter } from '@flux-gate/core';

const limiter = new SlidingWindowLogLimiter({ limit: 100, windowMs: 60_000 }); // 100 per minute, exactly

Sliding window counter

Keeps just two counters, for the previous and the current window, and estimates the sliding total as previous * (fraction of previous still in view) + current. This is the approach Cloudflare describes for their rate limiting. It is cheap and has no sharp edge, but the estimate assumes traffic was spread evenly through the previous window, so it can be off when traffic is bunched.

import { SlidingWindowCounterLimiter } from '@flux-gate/core';

const limiter = new SlidingWindowCounterLimiter({ limit: 100, windowMs: 60_000 });

Fixed window

Cuts time into slices of windowMs and counts requests per key per slice, starting from zero in each new slice. The simplest and cheapest, with the edge burst described above.

import { FixedWindowLimiter } from '@flux-gate/core';

const limiter = new FixedWindowLimiter({ limit: 100, windowMs: 60_000 });

Charging by cost, not by request

Requests are not equal. One LLM call can use 100 times the tokens of another, and one export can scan a million rows. Pass cost to make a limiter count units instead of requests, and size the limit in the same unit:

// A burst of 10 units, refilled at 2 per second.
const limiter = new TokenBucketLimiter({ capacity: 10, refillPerSecond: 2 });

limiter.consume('k', { cost: 4 });   // { allowed: true,  limit: 10, remaining: 6, resetMs: 2000 }
limiter.consume('k', { cost: 7 });   // { allowed: false, ..., remaining: 6, retryAfterMs: 500 }   only 6 left
limiter.consume('k', { cost: 11 });  // { allowed: false, ..., remaining: 6 }                        no retryAfterMs

The rules:

• A request of cost N is admitted only if N requests of cost 1 would all be admitted in a row. So every algorithm's guarantees carry over unchanged, just counted in units.
• cost must be a positive whole number. Anything else throws a RangeError (a bad cost is a bug in your code, so it's not hidden).
• A denial charges nothing.
• A request that is bigger than the whole limit gets a denial without retryAfterMs, because it can never fit. In HTTP terms, answer it with 413, not 429: clients and SDKs retry a 429 forever.

Testing without waiting

Every limiter's consume() accepts an optional now in milliseconds since the epoch. It defaults to Date.now(). Passing it makes tests exact: no sleeping, no fake timers, no flaky timing.

import { expect, test } from 'vitest';
import { TokenBucketLimiter } from '@flux-gate/core';

test('a caller earns one token back per second', () => {
  const limiter = new TokenBucketLimiter({ capacity: 1, refillPerSecond: 1 });
  const t0 = 1_000_000;

  expect(limiter.consume('a', { now: t0 }).allowed).toBe(true);
  expect(limiter.consume('a', { now: t0 }).allowed).toBe(false);
  expect(limiter.consume('a', { now: t0 + 500 }).allowed).toBe(false);  // half a token
  expect(limiter.consume('a', { now: t0 + 1000 }).allowed).toBe(true);  // a whole one
});

You can also use it to see why algorithms differ. This is the same traffic, three requests just before a window edge and three just after, into a limit of 3 per second:

const traffic = [999, 999, 999, 1000, 1000, 1000];

const fixed = new FixedWindowLimiter({ limit: 3, windowMs: 1000 });
traffic.map((now) => fixed.consume('a', { now }).allowed);
// [true, true, true, true, true, true]     6 admitted in 2 ms: double the limit

const log = new SlidingWindowLogLimiter({ limit: 3, windowMs: 1000 });
traffic.map((now) => log.consume('a', { now }).allowed);
// [true, true, true, false, false, false]  never more than 3 in any second

Using it in your app

Pick the setup that matches where your app runs.

One server, with middleware. Add @flux-gate/http for Express, Fastify, Hono, or anything built on the Fetch API (Next.js, Bun, Deno, Cloudflare Workers). It sends the standard X-RateLimit-* and Retry-After headers, answers 429 or 413, and supports plans and metrics:

import { TokenBucketLimiter } from '@flux-gate/core';
import { fluxgate } from '@flux-gate/http/express';

app.use('/api', fluxgate({ limiter: new TokenBucketLimiter({ capacity: 20, refillPerSecond: 2 }) }));

Several servers, one shared limit. The limiter above lives in one process's memory, so three servers behind a load balancer would each allow the full limit (we measured 2.93x the limit in that setup). Swap in @flux-gate/redis and every instance counts against the same bucket, atomically:

import { connectRedis, RedisTokenBucket } from '@flux-gate/redis';

const redis = await connectRedis({ url: process.env.REDIS_URL });
const limiter = new RedisTokenBucket(redis, { capacity: 20, refillPerSecond: 2 });

await limiter.consume('user:42'); // same result shape, but a Promise

No web framework at all. The limiter is just an object. Call it from a WebSocket handler, a queue worker, a CLI or a cron job:

socket.on('message', (data) => {
  if (!limiter.consume(`conn:${socket.id}`).allowed) return socket.close(1008, 'rate limit'); // 1008: policy violation
  handle(data);
});

Things to know

• State is in memory and per process. Two copies of your app count separately. Use @flux-gate/redis for a shared limit.
• Idle keys are never evicted. Each distinct key you ever pass stays in memory until the process exits. That is fine for a bounded set (API keys, tenants, users). If keys are effectively unbounded (every IP address on the internet, say), prefer @flux-gate/redis, where keys expire on their own, or limit by a bounded identifier.
• It is ESM only. From a CommonJS file, load it with const { TokenBucketLimiter } = await import('@flux-gate/core');. Requires Node.js 20 or newer.
• Keys are just strings. Namespace them (user:42, ip:203.0.113.7, tenant:acme:export) so different kinds of callers can never collide, and keep them short.
• Separate limiters for separate limits. A login limiter and an API limiter are two instances with two configs. Don't try to share one.
• resetMs depends on the algorithm. It is always "until this key has nothing left on record", but that means different things: a full bucket for the token bucket and GCRA, the end of the current slice for the fixed window, and the moment the last remembered request leaves the window for the two sliding windows.

API reference

import {
  // The limiters
  TokenBucketLimiter,
  GcraLimiter,
  SlidingWindowLogLimiter,
  SlidingWindowCounterLimiter,
  FixedWindowLimiter,

  // Types
  type RateLimiter,        // { consume(key, options?): RateLimitResult }
  type AsyncRateLimiter,   // the same, returning a Promise (for stores like Redis)
  type AnyRateLimiter,     // either one
  type RateLimitResult,
  type ConsumeOptions,     // { cost? }
  type LocalConsumeOptions,// { cost?, now? }
  type TokenBucketState,   // { tokens, lastRefill }, for tokenBucketStep
  type TokenBucketConfig,  // { capacity, refillPerSecond }
  type GcraConfig,         // { burst, ratePerSecond }
  type SlidingWindowLogConfig, type SlidingWindowCounterConfig, type FixedWindowConfig, // { limit, windowMs }
} from '@flux-gate/core';

| Member | Description | |---|---| | limiter.consume(key, options?) | Decide and record. options.cost (default 1, a positive whole number) and options.now (default Date.now()). Returns a RateLimitResult. | | tokenBucketStep(state, config, now, cost?) | The token bucket's decision as a pure function: the old TokenBucketState (or undefined for a new key) in, { state, result } out. For building your own store: you load the state, call this, and save the new state. | | resolveCost(cost) | The validation consume() applies to cost. |

To write your own limiter (a different store, a custom rule), implement RateLimiter or AsyncRateLimiter. Anything with that consume method plugs into @flux-gate/http unchanged.

TypeScript types ship with the package; there is nothing extra to install.

The rest of FluxGate

@flux-gate/core is one of five packages. Install only what you need.

| Package | What it adds | |---|---| | @flux-gate/core | This package. Five algorithms, in memory, no dependencies. | | @flux-gate/redis | Limits shared across servers: an atomic Redis token bucket, a leased bucket that cuts Redis calls by about 18x, and a Redis client that fails fast instead of hanging. | | @flux-gate/http | Middleware for Express, Fastify, Hono and the Fetch API, with standard headers, per-plan tiers, timeouts, fail-open or fail-closed, and Prometheus metrics. | | @flux-gate/client | A client for the FluxGate service, for sharing limits across many apps and languages. | | @flux-gate/server | The deployable service: POST /v1/check, /metrics for Prometheus, configured by environment variables. |

The design reasoning, the race condition a naive Redis limiter has (and how it is fixed), and all the measurements are in the FluxGate repository: DESIGN.md explains how and why, REPORT.md has the method and results. Found a bug or have a question? Open an issue.

MIT © Krish Vaghasia