koa-grounded

A distributed rate-limit middleware for Koa 2, inspired by Dcard's intern preliminary project.

Quick Start

Install

$ yarn add koa-grounded

Basic Usage

const Koa = require('koa');
const Grounded = require('koa-grounded');

const app = new Koa();

// Remember to set app.proxy flag if you are using under a reverse proxy
app.proxy = true;

const RateLimiter = Grounded({
  partitionKey: 'grounded',
  ratelimit: 1000,
  globalEXP: 60 * 60 * 1000 * 1000, // expiration time in μs
  timeout: 2000,
  cacheSize: 500,
  dbStr: 'redis://127.0.0.1:6379/',
  verbose: true,
});

// Grounded Middleware
app.use(RateLimiter);

// Ping Pong
app.use(async (ctx) => {
  ctx.body = 'Pong!';
});

// Using IPv4 instead of IPv6;
app.listen(4000, '0.0.0.0');

console.log('listening on port 4000');

See the document #API for further informations.

API

| Param | Type | Description | |--------------|-----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | partitionKey | string | A partition key for current Rate-Limiter to listen, will create ${partitionKey}-exp, ${partitionKey}-remaining and ${partitionKey}-ban keys on Redis Instance, and subscribe to ${partitionKey}-ban and ${partitionKey}-unban channels | | ratelimit | number | Ratelimit for each user's session | | globalEXP | number | Expiration time for user's ratelimit session, in microseconds(10^-6 seconds) unit | | timeout | number | Worker-Redis synchronization intervals, in miliseconds(10^-3 seconds) unit, it is suggested to modify the value as fast as the worker localQueue size reaches MTU size | | cacheSize | number | Maxmimum key size stored on local LRU cache | | dbStr | string | Connection string to the Redis instance, see luin/ioredis#connect-to-redis for further information | | verbose | boolean | Showing access log informations or not |

Running tests

$ yarn test

Overview

Concept

Introduction

Since Redis is fast enough for its in-memory data operations, the bottleneck of a Redis connection is the Round-Trip Time(RTT), which may dramatically affects throughputs of services having Redis as the centralized datastore.

This approach implemented a Eventually consistency and Availability-Partition tolerance(AP) approach using pipelined Lua scripts, LRU cache and Pub/Sub to optimize the throughput of the Rate-Limit service, and it is also capable of:

• [X] sharing states among all workers
• [X] key-space partitioning
• [X] Ratelimiting

As a result, we can achieve 10x faster than normal non-pipelined Redis approach with such optimization(See #Benchmark for details).

Implementation

WIP

Benchmark

One Worker, local Redis

(Redis Instance on Intel Core i7 9850H, 16GB RAM, macOS 10.15.3)

Ping RTT

--- localhost ping statistics ---
20 packets transmitted, 20 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.050/0.093/0.113/0.017 ms

using koajs/ratelimit

➜  ~ wrk -t12 -c1200 -d5s
Running 5s test
  12 threads and 1200 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    34.73ms   20.94ms 627.03ms   81.09%
    Req/Sec   760.13    236.27     1.42k    74.61%
  39505 requests in 5.07s, 11.05MB read
  Socket errors: connect 0, read 623, write 0, timeout 0
  Non-2xx or 3xx responses: 38505
Requests/sec:   7794.90
Transfer/sec:      2.18MB

our implementation

➜  ~ wrk -t12 -c1200 -d5s
Running 5s test
  12 threads and 1200 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    13.74ms   11.99ms 479.43ms   98.00%
    Req/Sec     1.53k   592.69     3.90k    77.36%
  84128 requests in 5.10s, 18.43MB read
  Socket errors: connect 0, read 529, write 0, timeout 0
  Non-2xx or 3xx responses: 83128
Requests/sec:  16500.27
Transfer/sec:      3.61MB

One Worker, remote Redis

(Redis Instance located on TANet, vSphere6.7, 1vCPU 2GB RAM, CentOS7 + docker 19.03)

Ping RTT

--- Remote-Redis ping statistics ---
20 packets transmitted, 20 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 19.858/67.130/250.806/61.965 ms

using koajs/ratelimit

➜  ~ wrk -t12 -c1200 -d5s
Running 5s test
  12 threads and 1200 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   129.69ms   56.67ms 632.29ms   71.43%
    Req/Sec   155.99     71.33   323.00     65.16%
  7712 requests in 5.10s, 2.10MB read
  Socket errors: connect 0, read 625, write 0, timeout 0
  Non-2xx or 3xx responses: 6712
Requests/sec:   1513.19
Transfer/sec:    422.69KB

our implementation

➜  ~ wrk -t12 -c1200 -d5s
Running 5s test
  12 threads and 1200 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    16.75ms   12.07ms 449.36ms   76.08%
    Req/Sec     1.48k   526.48     3.13k    74.82%
  81097 requests in 5.10s, 17.76MB read
  Socket errors: connect 0, read 613, write 0, timeout 0
  Non-2xx or 3xx responses: 80097
Requests/sec:  15899.12
Transfer/sec:      3.48MB

Roadmap

• [ ] Worker-Threads
• [ ] Increase Unit Test Coverage
• [ ] Support for other Redis Client
• [ ] LUA script for cleaning expired keys on Redis

License

MIT