ai-key-pool

Quick Start

const pool = new KeyPool({
  cooldownMs: 30000,
  providers: [
    {
      name: "gemini",
      keys: ["geminiApiKey"],
      models: ["gemini-2.5-flash"],
    },
    {
      name: "openai",
      keys: ["key1", "key2"],
      models: ["gpt-4o", "gpt-4o-mini"],
    },
  ],
});

const result = await pool.executeWithFallback(
  ["openai", "gemini"],
  async ({ apiKey, model }) => {
    // your LLM call
  },
);

A resilient failover system for API keys, models, and LLM providers.

ai-key-pool is a TypeScript library that provides automatic API key rotation, model switching, and multi-provider failover for LLM APIs like OpenAI, Gemini, and OpenRouter.

It ensures your AI application keeps working even when:

• keys hit rate limits

• models fail or become unavailable

• providers throttle or break

• individual requests fail unpredictably

• Perfect for agentic workflows where each LLM request is stateless and can safely fail over across keys, models, or providers.

Architecture Diagram

Request
   ↓
Key failure?
   ↓
Rotate key
   ↓
Model failure?
   ↓
Switch model
   ↓
Provider failure?
   ↓
Fallback provider

Why this exists

Production LLM apps don’t fail because of logic.

They fail because of:

• rate limits
• quota exhaustion
• unstable providers
• invalid or deprecated models

Most apps handle this manually (if at all).

ai-key-pool automates this entire reliability layer.

Features

• 🔁 Automatic API key rotation per provider
• 🧠 Model fallback on failure
• 🌐 Multi-provider failover (OpenAI, Gemini, OpenRouter, etc.)
• ⏳ Key cooldown for rate limit handling
• 🔌 Smart retry pipeline (key → model → provider)
• 🪝 Lifecycle hooks for observability
• 🧩 Fully TypeScript-first

Installation

npm install ai-key-pool

API

KeyPool config

type KeyPoolConfig = {
  cooldownMs?: number;
  strategy?: "priority" | "sticky" | "round-robin";
  providers: ProviderConfig[];
};

Provider config

type ProviderConfig = {
  name: string;
  keys: string[];
  models: string[];
};

executeWithFallback

pool.executeWithFallback(
  fallbackOrder: string[],
  handler: (ctx) => Promise<any>
)

Strategies

• priority – (DEFAULT) Uses providers in fallback order for every request
• sticky – Reuses the last successful provider + model for subsequent requests
• round-robin – Cycles providers across new requests

Lifecycle Hooks

const pool = new KeyPool({
  cooldownMs: 1000, // In ms
  providers: [],
  onKeyFailure: ({ provider, apiKey, error }) => {},
  onModelSwitch: ({ provider, apiKey, error }) => {},
  onProviderSwitch: ({ provider, apiKey, error }) => {},
});

Works with

• OpenAI
• Gemini
• OpenRouter
• Anthropic
• Any custom provider with API key + model abstraction

Examples

Minimal Config

import OpenAI from "openai";
import { KeyPool } from "ai-key-pool";

const pool = new KeyPool({
  cooldownMs: 30000,
  providers: [
    {
      name: "openai",
      keys: ["key1", "key2"],
      models: ["gpt-4o"],
    },
  ],
});

const result = await pool.executeWithFallback(
  ["openai"],
  async ({ apiKey, model }) => {
    // OpenAI SDK Implementation
    const client = new OpenAI({ apiKey });

    return client.chat.completions.create({
      model: model!,
      messages: [{ role: "user", content: "Hello!" }],
    });
  },
);

console.log(result);

Provider fallback

// Example: automatic fallback across OpenAI → Gemini → OpenRouter
// Demonstrates provider switching + key rotation + failure resilience

import OpenAI from "openai";
import { KeyPool } from "ai-key-pool";
import { GoogleGenAI } from "@google/genai";
import { createOpenRouter } from "@openrouter/ai-sdk-provider";
import { generateText } from "ai";

// KeyPool handles:
// - provider routing
// - key rotation
// - automatic fallback on failure
const pool = new KeyPool({
  cooldownMs: 30000,
  providers: [
    {
      name: "gemini",
      keys: ["geminiApiKey"],
      models: ["gemini-2.5-flash"],
    },
    {
      name: "openai",
      keys: ["openApiKey1", "openApiKey2"],
      models: ["gpt-4o", "gpt-4o-mini"],
    },
    {
      name: "openrouter",
      keys: ["openRouterApiKey1", "openRouterApiKey2", "openRouterApiKey3"],
      models: ["gpt-4o", "gpt-4o-mini"],
    },
  ],
});

const prompt = "Explain the purpose of life in 2 lines.";

const runOpenAI = (apiKey: string, model: string) => {
  const client = new OpenAI({ apiKey });

  return client.chat.completions.create({
    model,
    messages: [{ role: "user", content: prompt }],
  });
};

const runGeminiAI = (apiKey: string, model: string) => {
  const ai = new GoogleGenAI({ apiKey });

  return ai.models.generateContent({
    model,
    contents: prompt,
  });
};

const runOpenrouterAI = (apiKey: string, model: string) => {
  const openrouter = createOpenRouter({ apiKey });

  return generateText({
    model: openrouter(model),
    prompt,
  });
};

const result = await pool.executeWithFallback(
  ["gemini", "openai", "openrouter"], // order of fallback priority

  async ({ apiKey, provider, model }) => {
    if (!model) throw new Error("Model missing");

    switch (provider) {
      case "openai":
        return runOpenAI(apiKey, model);

      case "gemini":
        return runGeminiAI(apiKey, model);

      case "openrouter":
        return runOpenrouterAI(apiKey, model);

      default:
        throw new Error(`Unknown provider: ${provider}`);
    }
  },
);

console.log(result);

Agentic workflows

See full example:

• Agentic workflow with fallback

License

MIT