smart-rate-limiter

Intelligent multi-server API rate limiting middleware for Express and Fastify, with:

• Token bucket, sliding window, and fixed window algorithms
• Atomic Redis operations via Lua for race-free counting across horizontally scaled instances (<1 ms enforcement latency)
• Semantic caching backed by Qdrant vector search at a 0.88 cosine-similarity threshold — eliminates redundant API and LLM calls
• 3-tier storage with graceful in-memory fallback and tiered free / pro / enterprise limits
• SQLite for config + analytics (RAM-cached after first read)
• Automated tests and npm releases via GitHub Actions CI/CD

npm install smart-rate-limiter

Quick start (zero-config)

No Redis, no Qdrant, no Docker. Just install and go.

import express from 'express';
import { rateLimiter } from 'smart-rate-limiter';

const app = express();
app.use(rateLimiter({ max: 100, window: '1m' }));

app.get('/api/search', (_req, res) => res.json({ results: [] }));
app.listen(3000);

Response headers automatically added:

X-RateLimit-Limit: 100
X-RateLimit-Remaining: 99
X-RateLimit-Reset: 1735000060

Production (distributed, multi-instance)

Point at a shared Redis, and every instance behind your load balancer sees the same counters.

app.use(rateLimiter({
  max: 100,
  window: '1m',
  redis: process.env.REDIS_URL,             // 'redis://localhost:6379'

  routeCost: {
    '/api/search': 5,                       // costs 5 tokens
    '/api/health': 0,                       // free
  },

  tiers: {
    free:       { max: 100,   window: '1m' },
    pro:        { max: 1000,  window: '1m' },
    enterprise: { max: 10000, window: '1m' },
  },
  identifyTier: (req) => req.user?.plan ?? 'free',
}));

Full (semantic cache enabled)

Add Qdrant for semantic deduplication. Similar queries share cache entries, saving rate-limit quota and API costs.

docker run -p 6333:6333 qdrant/qdrant

app.use(rateLimiter({
  max: 100,
  window: '1m',
  redis: process.env.REDIS_URL,

  semanticCache: {
    qdrantUrl: process.env.QDRANT_URL,      // 'http://localhost:6333'
    embeddingModel: 'minilm',
    threshold: 0.88,                        // similarity 0-1
    ttl: '24h',
    routes: ['/api/search', '/api/recommend'],
  },
}));

// Two similar requests share one cache entry:
//   GET /api/search?q=hotels+in+mumbai   → API hit, cached
//   GET /api/search?q=mumbai+hotels      → cache hit (similarity 0.92)
//                                          rate limit NOT consumed

Fastify

import Fastify from 'fastify';
import { fastifyRateLimiter } from 'smart-rate-limiter';

const app = Fastify();
await app.register(fastifyRateLimiter({ max: 100, window: '1m' }));

Architecture

Three databases, three access patterns — none can replace the others:

| Store | Role | Why it's chosen | | ------ | ------------------------------------------- | ----------------------------------------------------- | | Redis | Real-time counters + Lua atomic decrement | Shared across instances, sub-millisecond, race-free | | SQLite | Route/tier config + analytics | Zero-config, embedded, RAM-cached microsecond reads | | Qdrant | Vector search over cached request/response | Only DB purpose-built for cosine similarity search |

Request lifecycle

1. Request arrives at your Express/Fastify app.
2. Middleware intercepts before your route handler runs.
3. SQLite lookup: which rule applies to this user + route?
4. Redis atomic Lua check: does this user have tokens remaining?
5. If the route has semanticCache enabled: embed the request body → search Qdrant → similarity above threshold? Return cache hit, skip the real handler.
6. If all checks pass: next() — your handler runs.
7. Response intercepted on the way out: store embedding + response in Qdrant for future hits.
8. X-RateLimit-*, Retry-After, and X-Cache-Hit headers attached.

The core algorithm — Redis Lua

The most critical piece of the system. This Lua script runs atomically inside Redis — no other command can execute between any of its steps. That is what prevents race conditions.

-- scripts/lua/tokenBucket.lua
local key       = KEYS[1]
local capacity  = tonumber(ARGV[1])
local rate_ps   = tonumber(ARGV[2])
local cost      = tonumber(ARGV[3])
local now       = tonumber(ARGV[4])

local state       = redis.call('HMGET', key, 'tokens', 'last_refill')
local tokens      = tonumber(state[1]) or capacity
local last_refill = tonumber(state[2]) or now

local elapsed = (now - last_refill) / 1000
tokens = math.min(capacity, tokens + elapsed * rate_ps)

local allowed = 0
if tokens >= cost then
  tokens = tokens - cost
  allowed = 1
end
redis.call('HMSET', key, 'tokens', tokens, 'last_refill', now)
redis.call('PEXPIRE', key, math.ceil((capacity / rate_ps) * 1000))
return { allowed, math.floor(tokens), now + math.ceil((capacity - tokens) / rate_ps * 1000) }

Sliding-window and fixed-window scripts live in scripts/lua/.

Folder structure

smart-rate-limiter/
├── src/
│   ├── index.ts              ← main export
│   ├── middleware/
│   │   ├── rateLimiter.ts    ← core middleware logic
│   │   ├── interceptor.ts    ← response interception for cache store
│   │   └── fastify.ts        ← fastify plugin adapter
│   ├── algorithms/
│   │   ├── tokenBucket.ts
│   │   ├── slidingWindow.ts
│   │   ├── fixedWindow.ts
│   │   └── luaScripts.ts     ← inline Lua for Redis storage
│   ├── storage/
│   │   ├── memory.ts         ← in-memory fallback
│   │   ├── redis.ts          ← Redis client + Lua
│   │   ├── sqlite.ts         ← SQLite config + analytics
│   │   └── qdrant.ts         ← vector store
│   ├── semantic/
│   │   ├── embedder.ts       ← text → vector
│   │   └── cache.ts          ← semantic cache API
│   ├── config/
│   │   └── defaults.ts
│   └── types.ts
├── scripts/
│   └── lua/                  ← published raw Lua scripts
├── tests/                    ← Jest tests (44 tests, >80% coverage)
├── examples/
│   ├── express-basic/
│   └── fastify-basic/
├── Dockerfile
├── docker-compose.yml        ← app + Redis + Qdrant
└── docker-compose.dev.yml    ← Redis + Qdrant only (for local dev)

Docker Compose (full stack)

docker compose up -d
# app       → http://localhost:3000
# redis     → localhost:6379
# qdrant    → http://localhost:6333

Dev-only backing services (no app image build):

docker compose -f docker-compose.dev.yml up -d

API reference

rateLimiter(options)

Returns an Express-compatible middleware with an attached .close() method for graceful shutdown.

| Option | Type | Default | Description | | ---------------- | ----------------------------------------------- | ------------- | ------------------------------------------------------- | | max | number | 100 | Max tokens/requests per window. | | window | '1m' \| '30s' \| number | '1m' | Rolling window duration. | | algorithm | 'token-bucket' \| 'sliding-window' \| 'fixed-window' | 'token-bucket' | Which algorithm to run. | | redis | string | - | Redis URL. Omit for in-memory. | | sqlite | string | - | SQLite file path. Omit for in-memory. | | routeCost | Record<string, number> | - | Per-route token cost. 0 skips counting. | | tiers | Record<string, { max, window }> | - | Per-tier limits. | | keyGenerator | (req) => string | IP-based | Compute the caller's identity. | | identifyTier | (req) => string | () => 'free'| Compute the caller's tier. | | semanticCache | SemanticCacheOptions | - | Enable Qdrant-backed semantic cache. | | headers | boolean | true | Attach X-RateLimit-* headers. | | statusCode | number | 429 | HTTP status when denied. | | message | string \| object | text | Response body when denied. | | onRequest | (ctx) => void | - | Called on every decision. | | onLimitReached | (ctx) => void | - | Called when a request is denied. |

Hosting

• Railway: push to GitHub, click "Add Redis" — Railway injects REDIS_URL. For Qdrant, add a Docker service.
• AWS EC2 / DigitalOcean: apt install redis-server, docker run qdrant/qdrant, pm2 start app.js.
• Docker Compose: docker compose up -d — the full stack in one command.
• Vercel / Netlify: use Upstash for serverless Redis. Semantic cache (Qdrant) is not supported on serverless.

License

MIT © Parichaye Grover