ttc_rate_limit

A rate limiting library with worker pool support for parallel timeout management.

Features

• Core Rate Limiting: Token bucket and sliding window algorithms via RateLimiter
• Cluster Management: Coordinate multiple rate limiters with Cluster for load balancing and failover

Installation

npm install ttc_rate_limit

Usage

Single Rate Limiter

import { RateLimiter } from 'ttc_rate_limit';

// Create a rate limiter for API calls
const apiLimiter = new RateLimiter({
  id: 'api-limiter',
  request: 100,          // 100 requests
  per: 'minute',         // per minute
  mode: 'spread',        // spread evenly over time
  cb: async (params) => {
    // Your API call here
    return await fetch('https://api.example.com', params);
  },
});

// Make a request (automatically rate-limited)
const result = await apiLimiter.invoke({ url: '/api/data' });

Rate Limiting Modes

• spread: Distributes requests evenly over the time period. For 100 requests per minute, sends ~1.67 requests per second.
• burst: Allows requests up to the limit instantly, then waits for the full time window to reset. For 100/minute, sends 100 requests at once, then waits 60 seconds.
• hybrid: Combines burst and spread. Uses hybridRatio to divide the limit into smaller bursts. For 100/minute with hybridRatio: 10, sends 10 requests every 6 seconds.

// Burst mode example
const burstLimiter = new RateLimiter({
  id: 'burst-limiter',
  request: 100,
  per: 'minute',
  mode: 'burst',
  cb: async (input) => apiCall(input),
});

// Hybrid mode example
const hybridLimiter = new RateLimiter({
  id: 'hybrid-limiter',
  request: 100,
  per: 'minute',
  mode: 'hybrid',
  hybridRatio: 10,  // 10 bursts per 6 seconds
  cb: async (input) => apiCall(input),
});

Cluster Management

Use Cluster to manage multiple rate limiters and distribute tasks among them using load balancing strategies.

import { Cluster } from 'ttc_rate_limit';

type Task = { modelId: string; chatId: string; };

const cluster = new Cluster<Task, string>('roundrobin');

// Add multiple rate limiters
cluster.addOption({
  id: 'api-1',
  request: 100,
  per: 'minute',
  mode: 'hybrid',
  hybridRatio: 10,
  cb: async (task) => {
    // Process task with API 1
    return `Processed by API 1: ${task.modelId} - ${task.chatId}`;
  },
});

cluster.addOption({
  id: 'api-2',
  request: 50,
  per: 'minute',
  mode: 'spread',
  cb: async (task) => {
    // Process task with API 2
    return `Processed by API 2: ${task.modelId} - ${task.chatId}`;
  },
});

// Listen for events
cluster.on('completed', (data) => {
  console.log('Task completed:', data.response);
});

cluster.on('error', (data) => {
  console.error('Task error:', data.response);
});

// Invoke tasks (will be distributed among limiters)
for (let i = 0; i < 10; i++) {
  cluster.invoke({ modelId: 'gpt-4', chatId: `chat-${i}` });
}

Cluster Strategies

• random: Randomly select a limiter for each task.
• roundrobin: Cycle through limiters sequentially.

License

License

MIT