omniai-router (Advanced Edition)

🚀 Production-Grade AI Orchestration SDK — Advanced unified AI API router with multiple models, smart routing, streaming, cost tracking, and automatic fallback system.

Features

✨ Multiple Models — Support for 10+ AI models across 4 providers
🎯 Smart Routing — Mode-based selection (fast, cheap, smart, balanced)
🌊 Real-Time Streaming — Stream text generation responses
💰 Cost Tracking — Automatic token counting and cost estimation
📊 Token Usage — Real-time token tracking across sessions
🔄 Automatic Fallback — Seamlessly switch models on failure
🔁 Retry System — Exponential backoff with configurable attempts
⏱️ Timeout Handling — Request timeout with AbortController
🔐 Secure — Environment variables only, zero hardcoded keys
🧩 Production-Ready — Clean architecture, fully typed

Installation

npm install omniai-router

Quick Start

1. Set Up Environment Variables

Create a .env file in your project:

GEMINI_API_KEY=your_gemini_key_here
TOGETHER_API_KEY=your_together_key_here
OPENAI_API_KEY=your_openai_key_here
DEEPSEEK_API_KEY=your_deepseek_key_here

You only need to configure the providers you plan to use. omniai-router will use those available.

2. Basic Usage

import { generateText } from "omniai-router";

const result = await generateText({
  prompt: "Explain quantum computing in simple terms",
});

console.log(result.text);
// Output: { text: "...", provider: "gemini", model: "gemini-flash", durationMs: 1234 }

3. With Options

const result = await generateText({
  prompt: "Write a poem",
  mode: "smart", // fast, cheap, smart, balanced
  maxTokens: 500,
  temperature: 0.8,
  debug: true, // Enable logging
  timeout: 60000, // 60 second timeout
  retryAttempts: 3, // Retry up to 3 times
});

console.log(`Cost: ${result.cost}`);

Supported Models

Google Gemini

| Model | Type | Speed | Cost | | -------------- | ----- | ------- | --------- | | gemini-flash | fast | Fastest | Free tier | | gemini-pro | smart | Fast | Low |

OpenAI

| Model | Type | Speed | Cost | | --------------- | -------- | ------ | ---- | | gpt-4-turbo | smart | Medium | High | | gpt-3.5-turbo | balanced | Fast | Low |

Together AI

| Model | Type | Speed | Cost | | -------------- | ----- | ------- | -------- | | mixtral-8x7b | cheap | Fast | Very Low | | llama-2-7b | cheap | Fastest | Very Low |

DeepSeek

| Model | Type | Speed | Cost | | --------------- | ----- | ----- | -------- | | deepseek-chat | cheap | Fast | Very Low |

API Reference

generateText(options)

Main function for text generation.

Options

{
  prompt: string,                                    // Required
  model?: string,                                    // Specific model name
  mode?: "fast" | "cheap" | "smart" | "balanced",   // Default: "balanced"
  fallback?: boolean,                                // Default: true
  maxTokens?: number,                                // Default: 1000
  temperature?: number,                              // Default: 0.7, Range: 0-2
  timeout?: number,                                  // Default: 30000ms
  retryAttempts?: number,                            // Default: 2
  debug?: boolean,                                   // Default: false
  sessionId?: string,                                // For token tracking
}

Returns

{
  text: string,                // Generated text
  provider: string,            // Which provider was used
  model: string,               // Which model was used
  tokensUsed: {
    input: number,             // Input tokens
    output: number,            // Output tokens
    total: number,             // Total tokens
  },
  requestId: string,           // Request identifier
  mode: string,                // Mode used
  durationMs: number,          // Request duration
}

streamText(options)

Stream text generation for real-time responses.

import { streamText, collectStream } from "omniai-router";

// Stream chunks
for await (const chunk of streamText({ prompt: "..." })) {
  process.stdout.write(chunk.text);
}

// Or collect all at once
const result = await collectStream(streamText({ prompt: "..." }));
console.log(result.text);

getAvailableModelsInfo()

Get all available models based on configured API keys.

import { getAvailableModelsInfo } from "omniai-router";

const models = getAvailableModelsInfo();
console.log(models);

Routing Modes

Fast (Optimized for Speed)

await generateText({
  prompt: "...",
  mode: "fast", // Uses: gemini-flash → llama-2-7b
});

Cheap (Optimized for Cost)

await generateText({
  prompt: "...",
  mode: "cheap", // Uses: mixtral → llama-2 → deepseek
});

Smart (Optimized for Quality)

await generateText({
  prompt: "...",
  mode: "smart", // Uses: gpt-4 → gemini-pro → mixtral
});

Balanced (Default)

await generateText({
  prompt: "...",
  mode: "balanced", // Mix of all: fastest + best + cheapest
});

Specific Model Selection

Use exact model name for guaranteed provider:

// Always use GPT-4
const result = await generateText({
  prompt: "Complex analysis task",
  model: "gpt-4-turbo",
  fallback: true, // Fall back if GPT-4 fails
});

Streaming Examples

Basic Streaming

import { streamText } from "omniai-router";

for await (const chunk of streamText({
  prompt: "Write a story",
  mode: "fast",
})) {
  process.stdout.write(chunk.text);
}

Streaming with Buffer

import { streamText, collectStream } from "omniai-router";

const stream = streamText({
  prompt: "Explain AI",
  mode: "smart",
});

const full = await collectStream(stream);
console.log(`Generated ${full.text.length} characters`);

Token & Cost Tracking

Real-Time Tracking

import {
  generateText,
  globalTokenTracker,
  globalCostEstimator,
} from "omniai-router";

const sessionId = "session_123";

await generateText({
  prompt: "First request",
  sessionId,
});

await generateText({
  prompt: "Second request",
  sessionId,
});

const stats = globalTokenTracker.getStats(sessionId);
console.log(`Total tokens: ${stats.totalTokens}`);

const costs = globalCostEstimator.getCostBreakdown();
console.log(`Total cost: $${costs.totalCost}`);

Debug Logging

Enable detailed logging:

const result = await generateText({
  prompt: "...",
  debug: true,
});

// Output:
// [omniai-router][req_123] ▶ Generating text...
// [omniai-router][req_123] 🎯 Selected model: gemini-flash
// [omniai-router][req_123] ⏳ Trying model: gemini-flash
// [omniai-router][req_123] ✅ Success with: gemini-flash

Environment Setup

Get API Keys

| Provider | Sign Up Link | Free Tier | | --------------- | ---------------------------------------- | ------------ | | Gemini | https://makersuite.google.com/app/apikey | ✅ Yes | | Together AI | https://www.together.ai/ | ✅ Yes | | OpenAI | https://platform.openai.com/api-keys | ✅ $5 credit | | DeepSeek | https://platform.deepseek.com/ | ✅ Yes |

Load Environment Variables

Using dotenv (Recommended)

npm install dotenv

import dotenv from "dotenv";
dotenv.config();

import { generateText } from "omniai-router";

const res = await generateText({ prompt: "Test" });

Node.js with --env-file (v20.6+)

node --env-file=.env app.js

Error Handling

import {
  generateText,
  ValidationError,
  ModelNotAvailableError,
  NoAvailableProvidersError,
  TimeoutError,
} from "omniai-router";

try {
  const result = await generateText({ prompt: "..." });
} catch (error) {
  if (error instanceof ValidationError) {
    console.error("Invalid input:", error.message);
  } else if (error instanceof ModelNotAvailableError) {
    console.error("Model not available:", error.model);
  } else if (error instanceof NoAvailableProvidersError) {
    console.error("No providers configured");
  } else if (error instanceof TimeoutError) {
    console.error(`Timeout after ${error.timeoutMs}ms`);
  }
}

Advanced Usage

Custom Token Tracker

import { createTokenTracker } from "omniai-router";

const tracker = createTokenTracker();
const sessionId = tracker.createSession();

// Track tokens manually
tracker.trackTokens(sessionId, 100, 50);

const stats = tracker.getStats(sessionId);

Custom Cost Estimator

import { createCostEstimator } from "omniai-router";

const estimator = createCostEstimator();
const estimation = estimator.estimateCost("gpt-4-turbo", 150, 75);

console.log(`Cost: $${estimation.totalCost}`);

Retry Configuration

import { executeWithRetry } from "omniai-router";

const result = await executeWithRetry(
  async () => {
    return generateText({ prompt: "..." });
  },
  {
    maxAttempts: 5,
    initialDelayMs: 200,
    maxDelayMs: 10000,
    backoffMultiplier: 1.5,
    debug: true,
  },
);

Timeout Control

import { executeWithTimeout } from "omniai-router";

try {
  const result = await executeWithTimeout(
    () => generateText({ prompt: "..." }),
    10000, // 10 second timeout
  );
} catch (error) {
  console.error("Operation timed out");
}

Best Practices

✅ Do:

• Start with mode: "balanced" for general use
• Use debug: true during development
• Set appropriate timeout values (30s default is good)
• Use sessionId to track related requests
• Implement error handling with specific error types

❌ Don't:

• Hardcode API keys (use .env)
• Use mode without fallback for critical requests
• Set very low timeout values (<1s)
• Ignore rate limit errors (implement backoff)
• Disable fallback in production

Architecture

src/
├── core/
│   ├── router.js          # Main orchestrator
│   ├── modelRegistry.js   # Model definitions
│   ├── tokenTracker.js    # Token tracking
│   ├── costEstimator.js   # Cost calculation
│   ├── fallback.js        # Fallback logic
│   └── config.js          # Configuration
├── providers/
│   ├── gemini.js
│   ├── openai.js
│   ├── together.js
│   └── deepseek.js
├── features/
│   ├── streaming.js       # Stream support
│   ├── retry.js           # Retry logic
│   └── timeout.js         # Timeout handling
├── utils/
│   ├── errors.js
│   ├── logger.js
│   └── helpers.js
└── index.js

Performance Tips

• Use streaming for long content — Real-time response better UX
• Cache results — Avoid duplicate requests
• Use fast mode for interactive apps — Lower latency
• Use cheap mode for batch processing — Lower cost
• Set sessionId for related requests — Better tracking
• Increase maxTokens only when needed — Reduces cost/time

Troubleshooting

"API key is missing"

# .env file missing API_KEY
GEMINI_API_KEY=your_actual_key

"No models available for mode"

// Check available models
const models = getAvailableModelsInfo();
console.log(models.map((m) => `${m.name} (${m.provider})`));

Slow responses

// Try faster mode
await generateText({
  prompt: "...",
  mode: "fast", // Instead of "balanced"
  model: "gemini-flash",
});

Rate limiting

// Increase retry attempts and timeouts
await generateText({
  prompt: "...",
  retryAttempts: 5,
  timeout: 60000,
});

Contributing

Contributions welcome! Areas for enhancement:

• New provider support
• Better cost estimation
• Caching layer
• Request queuing
• Rate limiting

License

MIT

Support

• 📚 GitHub Repo
• 🐛 Issues
• 💬 Discussions

Built for production. Used by thousands of developers.