api-throttle-llm
v1.0.0
Published
Token-bucket rate limiter for LLM API calls with per-model rate limits
Maintainers
Readme
api-throttle-llm
Token-bucket rate limiter for LLM API calls.
Installation
npm install api-throttle-llmUsage
const { RateLimiter } = require('api-throttle-llm');
const limiter = new RateLimiter();
if (limiter.tryAcquire('gpt-4o')) { /* make API call */ }
await limiter.acquire('gpt-4o'); // waits if neededLicense
MIT
