gemini-rotate

v0.1.2

Published

a month ago

A lightweight Node.js library for Google Gemini API key rotation, model fallback, and integrated LangSmith tracing.

0High
0Medium
0Low

jayeeed

gemini google genai api-key-rotation model-fallback langsmith tracing llm

Gemini Rotate

A lightweight Node.js library for Google Gemini API key rotation, valid model selection, and automatic fallback to secondary models on errors. Exact JavaScript port of the Python gemini-rotate library.

🚀 Features

✅ Automatic Key Rotation: Seamlessly rotates through a list of API keys when quota is exhausted (429), permission denied (403), or any other API error occurs.
🔄 Smart Model Fallback: Automatically falls back to a secondary model per-client when a primary model fails, then moves to the next API client.
🧠 Auto Thinking Management: Automatically configures reasoning budget for Gemini 3 and thinking models — enables full reasoning for long prompts or file inputs, and disables it for short prompts to save cost.
⚡ Async Support: Built on top of the @google/genai client, offering async generateContent and generateContentSync methods.
🛡️ Robust Error Handling: Catches API errors and rotates keys/models before throwing AllClientsFailed.
📝 Concise Logging: Logs only essential success/failure information to keep your console clean.
📊 Integrated LangSmith Tracing: Zero-setup wrapper integration with LangSmith. Automatically traces requests, attributes success to specific API clients and models, and logs accurate pricing and token metadata.

📦 Installation

npm install gemini-rotate

⚡ Quick Start

Configure Environment: Copy .env.example to .env and configure your API keys:

GEMINI_API_KEY_1="AIzaSy..."
GEMINI_API_KEY_2="AI3yhj..."
GEMINI_API_KEY_3="AIdf56..."

Run Code:

import "dotenv/config";
import { GeminiRotationClient } from "gemini-rotate";

const client = new GeminiRotationClient();
const response = await client.generateContent("Hello, Gemini!");
console.log(response.text);

📖 Usage Guide

Initialization

The client automatically loads API keys from your environment variables (GEMINI_API_KEY_1, GEMINI_API_KEY_2, etc.).

import { GeminiRotationClient } from "gemini-rotate";

const client = new GeminiRotationClient();

Generating Content

1. Async Text Generation

import "dotenv/config";
import { GeminiRotationClient } from "gemini-rotate";

async function generateText() {
  const client = new GeminiRotationClient();
  try {
    const response = await client.generateContent(
      "Explain quantum computing in 50 words."
    );
    console.log(`Generated text: ${response.text}`);
    console.log(`Client ID: ${response.clientId}`);
    console.log(`Model used: ${response.model}`);
  } catch (e) {
    console.error(`Error: ${e.message}`);
  }
}

generateText();

2. Sync-Named Generation (same as async in JS)

import "dotenv/config";
import { GeminiRotationClient } from "gemini-rotate";

async function generateTextSync() {
  const client = new GeminiRotationClient();
  try {
    const response = await client.generateContentSync(
      "Explain quantum computing in 50 words."
    );
    console.log(`Generated text: ${response.text}`);
  } catch (e) {
    console.error(`Error: ${e.message}`);
  }
}

generateTextSync();

Note: In Node.js, all I/O is inherently async. generateContentSync is provided for API naming parity with the Python library — both methods return a Promise.

3. Advanced: Structured Output

You can pass responseSchema and responseMimeType via the config parameter.

import "dotenv/config";
import { GeminiRotationClient } from "gemini-rotate";

async function generateRecipe() {
  const client = new GeminiRotationClient();

  try {
    const response = await client.generateContent(
      "Give me a recipe for chocolate cake.",
      {
        responseMimeType: "application/json",
        responseSchema: {
          type: "object",
          properties: {
            title: { type: "string" },
            ingredients: { type: "array", items: { type: "string" } },
            instructions: { type: "array", items: { type: "string" } },
          },
          required: ["title", "ingredients", "instructions"],
        },
      }
    );

    const recipe = JSON.parse(response.text);
    console.log(`Title: ${recipe.title}`);
    console.log(`Ingredients: ${recipe.ingredients.join(", ")}`);
  } catch (e) {
    console.error(`Error: ${e.message}`);
  }
}

generateRecipe();

4. Express.js Integration

import "dotenv/config";
import express from "express";
import { GeminiRotationClient } from "gemini-rotate";

const app = express();
app.use(express.json());

const client = new GeminiRotationClient();

app.post("/api/generate", async (req, res) => {
  try {
    const { prompt, config } = req.body;
    const response = await client.generateContent(prompt, config);

    res.json({
      text: response.text,
      model: response.model,
      clientId: response.clientId,
      usage: response.usageMetadata
        ? {
            inputTokens: response.usageMetadata.promptTokenCount,
            outputTokens: response.usageMetadata.candidatesTokenCount,
            totalCost: response.usageMetadata.totalCost,
          }
        : null,
    });
  } catch (e) {
    res.status(500).json({ error: e.message });
  }
});

app.listen(3000, () => console.log("Server running on port 3000"));

5. LangSmith Tracing Integration

gemini-rotate automatically wraps the content generation methods using LangSmith's traceable wrapper. This enables automatic tracing of your requests.

To activate tracing, configure your environment with the standard LangSmith variables:

export LANGCHAIN_TRACING_V2="true"   # or LANGSMITH_TRACING="true"
export LANGCHAIN_API_KEY="your-api-key"
# Optional:
export LANGCHAIN_PROJECT="my-gemini-project"

Note: Both LANGCHAIN_TRACING_V2 and LANGSMITH_TRACING environment variables are supported to enable tracing.

By default, the library traces requests with the tags ["gemini-rotate", "gemini", "nodejs"] and integration metadata. You can customize tags, metadata, or extra parameters by passing tracingExtra to the GeminiRotationClient constructor:

import "dotenv/config";
import { GeminiRotationClient } from "gemini-rotate";

const client = new GeminiRotationClient({
  tracingExtra: {
    tags: ["production", "chatbot"],
    metadata: {
      userId: "user_123",
    },
  },
});

const response = await client.generateContent(
  "Describe neural networks in one sentence."
);
console.log(response.text);

Parameters

Response Object

A successful response includes the standard @google/genai response fields, plus the following properties injected by the library:

Error Handling

The library exports two error classes:

import { GeminiRotationError, AllClientsFailed } from "gemini-rotate";

try {
  const response = await client.generateContent("...");
} catch (e) {
  if (e instanceof AllClientsFailed) {
    console.error("All API keys and models exhausted:", e.message);
  } else {
    throw e; // Re-throw unexpected errors
  }
}

⚙️ Configuration

1. The `.env` Format (Expected)

To configure the library, copy the provided .env.example file to .env in the root of your project.

# Required: Define your Gemini API keys using the pattern GEMINI_API_KEY_<number>
GEMINI_API_KEY_1="AIzaSy..."
GEMINI_API_KEY_2="AI3yhj..."
GEMINI_API_KEY_3="AIdf56..."

# Optional: Define your models in a valid JSON array format.
# The models will be processed in pairs (Primary -> Secondary fallback).
GEMINI_MODELS='["gemini-3.5-flash", "gemini-3.1-flash-lite"]'

# Optional: LangSmith Tracing integration config.
# Note: Leaving these variables empty or unconfigured
# will automatically disable tracing wrapper logic without throwing missing key errors.
LANGCHAIN_TRACING_V2="true"
LANGCHAIN_API_KEY="your-langsmith-api-key"
LANGCHAIN_PROJECT="gemini-rotate"

(Note: A single GEMINI_API_KEY environment variable is also supported as a fallback. If numbered keys like GEMINI_API_KEY_1 are present, they are guaranteed to rotate in sequential order. Duplicates are removed automatically.)

2. Model Priority Breakdown

You can customize the order in which models are attempted by setting GEMINI_MODELS in .env as shown above. The string MUST be a valid JSON array. The library processes models in Primary → Secondary pairs.

Default Behavior (if GEMINI_MODELS is not set):

Custom Configuration:

GEMINI_MODELS='["gemini-3.5-flash", "gemini-3.1-flash-lite", "gemini-3-flash-preview"]'

3. Automatic Thinking Config (Gemini 3 & Thinking Models)

For Gemini 3 and any model whose name contains "thinking", the library automatically manages the thinkingConfig unless you set it explicitly in config:

Short prompts (< ~100 tokens, no files): thinkingBudget is set to 0 to minimize cost.
Long prompts (≥ ~100 tokens) or prompts containing file/image parts: Full reasoning is enabled (no budget constraint).

You can always override this behavior by passing your own thinkingConfig in the config object:

const response = await client.generateContent("My prompt", {
  thinkingConfig: { thinkingBudget: 1024 },
});

🔍 How it Works

1. Execution Flow & Retries

The library iterates through model pairs in order. For each pair, it tries every API client in sequence. If the primary model fails for a given client, it immediately tries the secondary model with the same client before moving on to the next client.

graph TD
    Start[Start Request] --> IsTraced{LangSmith Enabled?}
    IsTraced -->|Yes| StartTrace[Start Trace Parent Run]
    IsTraced -->|No| LoopGroups
    StartTrace --> LoopGroups{Loop Model Groups}

    LoopGroups -->|"Group [Primary, Secondary]"| LoopClients{Loop API Clients}

    LoopClients -->|Next Client| AttemptPrimary[Attempt Primary Model]

    AttemptPrimary -->|Success| EnrichResponse[Enrich Response with clientId, model, cost]
    AttemptPrimary -->|API Error| CheckSecondary{Has Secondary Model?}

    CheckSecondary -->|Yes| AttemptSecondary[Attempt Secondary Model\nwith same client]
    CheckSecondary -->|No| NextClient[Move to Next Client]

    AttemptSecondary -->|Success| EnrichResponse
    AttemptSecondary -->|API Error| NextClient

    EnrichResponse --> LogTraceSuccess[Log Client/Model/Cost to Trace]
    LogTraceSuccess --> ReturnResponse[Return Response]

    NextClient -->|More Clients| LoopClients
    NextClient -->|Clients Exhausted| NextGroup[Next Model Group]
    NextGroup -->|More Groups| LoopGroups
    NextGroup -->|Groups Exhausted| LogTraceFailure[Log Failure to Trace]
    LogTraceFailure --> RaiseError[Throw AllClientsFailed]

    style Start fill:#f9f,stroke:#333,stroke-width:2px
    style ReturnResponse fill:#9f9,stroke:#333,stroke-width:2px
    style RaiseError fill:#f99,stroke:#333,stroke-width:2px

2. Tracing Pipeline Integration

If LangSmith tracing is enabled:

Parent Trace Context: The entire rotation execution is wrapped in a high-level parent run via traceable, allowing you to see the aggregate latency and outcome.
Outcome Attribution: The successfully resolved key and model are dynamically logged under the trace's metadata attributes (succeeded_client, succeeded_model, ls_model_name, ls_provider).
Precise Cost & Token Tracking: The library dynamically calculates pricing based on the model used and posts token counts and the calculated total USD cost directly to the trace outputs.

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.