hydra-aidirector

The official Node.js/TypeScript SDK for Hydra — a high-performance AI API gateway.

Why Hydra?

| Problem | Hydra Solution | |---------|----------------| | LLM outages break your app | 🔄 Automatic Failover – Seamless fallback between providers | | High latency & costs | ⚡ God-Tier Caching – Hybrid Redis + DB cache with AI-directed scoping | | Malformed JSON responses | 🛡️ Self-Healing AI – Auto-repair JSON, strip markdown, extract from prose | | Schema compliance issues | ✅ Strict JSON Mode – Force models to conform to your schema or fail | | No visibility into usage | 📊 Detailed Analytics – Track tokens, latency, and costs per model |

Installation

npm install hydra-aidirector
# or
pnpm add hydra-aidirector
# or
yarn add hydra-aidirector

Quick Start

import { Hydra } from 'hydra-aidirector';

const client = new Hydra({
  secretKey: process.env.HYDRA_SECRET_KEY!, // hyd_sk_...
  baseUrl: 'https://your-instance.vercel.app',
});

// Basic generation
const result = await client.generate({
  chainId: 'my-chain',
  prompt: 'Generate 5 user profiles as JSON',
});

if (result.success) {
  console.log(result.data.valid); // Parsed, schema-validated objects
}

Core Features

🔄 Automatic Failover

Define fallback chains in your dashboard. If Gemini fails, Hydra automatically tries OpenRouter, Claude, etc.

🛡️ Self-Healing JSON

LLMs sometimes return broken JSON. Hydra extracts and repairs it automatically:

const result = await client.generate({ chainId: 'x', prompt: 'Get user' });

if (result.meta.recovered) {
  console.log('JSON was malformed but healed!');
  console.log(result.data.healingReport);
  // [{ original: "{name: 'foo'", healed: { name: "foo" }, fixes: ["Added closing brace"] }]
}

✅ Strict JSON Mode vs Self-Healing

Hydra's self-healing JSON repair works in both modes. The difference is how the model behaves:

| Mode | Model Behavior | Healing Role | |------|----------------|--------------| | Strict (strictJson: true) | Model is forced to output pure JSON via native API constraints. Output is already clean. | Safety net — rarely needed since output is constrained. | | Non-Strict (strictJson: false) | Model outputs best-effort JSON (may include markdown, prose, or broken syntax). | Primary mechanism — extracts and repairs JSON from messy output. |

// Strict mode - model constrained to pure JSON
await client.generate({
  chainId: 'gemini-chain',
  prompt: 'Extract invoice data',
  schema: invoiceSchema,
  strictJson: true, // No markdown, no explanations
});

// Non-strict mode - flexible output, Hydra heals as needed
await client.generate({
  chainId: 'any-chain',
  prompt: 'Generate a creative story with metadata',
  schema: storySchema,
  strictJson: false, // Allow model to be creative, Hydra extracts JSON
});

Streaming (Recommended for Large Responses)

Process JSON objects as they arrive — perfect for UIs that need instant feedback:

await client.generateStream(
  {
    chainId: 'my-chain',
    prompt: 'Generate 100 product descriptions',
  },
  {
    onObject: (obj, index) => {
      console.log(`Object ${index}:`, obj);
      renderToUI(obj); // Render immediately!
    },
    onComplete: (result) => {
      console.log(`Done! ${result.objectCount} objects`);
    },
    onError: (error) => {
      console.error('Stream failed:', error);
    },
  }
);

Batch Generation

Process multiple prompts in parallel with automatic error handling:

const result = await client.generateBatch('my-chain', [
  { id: 'item1', prompt: 'Describe product A' },
  { id: 'item2', prompt: 'Describe product B' },
  { id: 'item3', prompt: 'Describe product C' },
]);

console.log(`Processed ${result.summary.succeeded}/${result.summary.total}`);

Caching

Cache Scope

Control how responses are cached:

// Global cache (shared across users - default)
await client.generate({ chainId: 'x', prompt: 'Facts', cacheScope: 'global' });

// User-scoped cache (private to authenticated user)
await client.generate({ chainId: 'x', prompt: 'My profile', cacheScope: 'user' });

// Skip cache entirely
await client.generate({ chainId: 'x', prompt: 'Random', cacheScope: 'skip' });

Cache Quality

Control the trade-off between cache hit rate and precision:

| Level | Behavior | |-------|----------| | STANDARD | Balanced fuzzy matching. Good for most cases. | | HIGH | Stricter matching. Higher quality hits, lower hit rate. | | MAX_EFFICIENCY | Aggressive matching. Maximum cost savings, less precision. |

await client.generate({
  chainId: 'my-chain',
  prompt: 'Generate report',
  cacheQuality: 'MAX_EFFICIENCY', // Maximize cache hits
});

AI-Directed Caching

The AI can override cache scope by including a _cache directive in its output:

{
  "data": { "...": "..." },
  "_cache": { "scope": "user" }
}

The directive is automatically stripped from your final response.

File Attachments

Upload documents for analysis. Hydra handles type detection and model compatibility:

import fs from 'fs';

const fileBuffer = fs.readFileSync('report.pdf');

const result = await client.generate({
  chainId: 'document-analysis',
  prompt: 'Summarize this document',
  files: [{
    data: fileBuffer.toString('base64'),
    filename: 'report.pdf',
    mimeType: 'application/pdf',
  }],
});

Request Cancellation

Cancel long-running requests with AbortSignal:

const controller = new AbortController();
setTimeout(() => controller.abort(), 5000); // Cancel after 5s

try {
  const result = await client.generate({
    chainId: 'my-chain',
    prompt: 'Long task',
    signal: controller.signal,
  });
} catch (error) {
  if (error instanceof TimeoutError) {
    console.log('Request was cancelled');
  }
}

Thinking Mode

Enable reasoning models to show their thought process:

const result = await client.generate({
  chainId: 'reasoning-chain',
  prompt: 'Solve this complex problem step by step',
  options: {
    thinkingMode: true,
  },
});

Webhooks

Register callbacks for async notifications:

// Register
await client.registerWebhook({
  requestId: 'req_123',
  url: 'https://your-domain.com/webhooks/hydra',
  secret: 'your-webhook-secret',
  retryCount: 3,
});

// Manage
const webhooks = await client.listWebhooks();
await client.updateWebhook('webhook_id', { retryCount: 5 });
await client.unregisterWebhook('webhook_id');

Configuration Reference

Client Options

| Option | Type | Default | Description | |--------|------|---------|-------------| | secretKey | string | required | Your API key (hyd_sk_...) | | baseUrl | string | http://localhost:3000 | API base URL | | timeout | number | 600000 | Request timeout in ms (10 min) | | maxRetries | number | 3 | Max retry attempts | | debug | boolean | false | Enable debug logging |

Generate Options

| Option | Type | Default | Description | |--------|------|---------|-------------| | chainId | string | required | Fallback chain ID | | prompt | string | required | The prompt to send | | schema | object | - | JSON schema for validation | | cacheScope | 'global' \| 'user' \| 'skip' | 'global' | Cache sharing behavior | | cacheQuality | 'STANDARD' \| 'HIGH' \| 'MAX_EFFICIENCY' | 'STANDARD' | Cache match precision | | strictJson | boolean | true | Force strict JSON mode | | signal | AbortSignal | - | Cancellation signal | | maxRetries | number | Client default | Override retries | | requestId | string | Auto-generated | Custom request ID | | files | FileAttachment[] | - | File attachments | | useOptimized | boolean | true | Use 3-step cost-saving flow | | noCache | boolean | false | Skip cache entirely |

API Methods

| Method | Description | |--------|-------------| | generate(options) | Generate content with fallback chain | | generateStream(options, callbacks) | Stream JSON objects in real-time | | generateBatch(chainId, items) | Process multiple prompts | | listModels() | List available AI models | | listChains() | List your fallback chains | | getUsage(options) | Get usage statistics | | health() | Check API health | | healthDetailed() | Get detailed component health | | registerWebhook(config) | Register async webhook | | unregisterWebhook(id) | Remove a webhook | | listWebhooks() | List all webhooks | | updateWebhook(id, updates) | Modify webhook config |

Error Handling

import {
  RateLimitError,
  TimeoutError,
  AuthenticationError,
  QuotaExceededError,
  WorkerError,
  FileProcessingError,
  isRetryableError,
} from 'hydra-aidirector';

try {
  const result = await client.generate({ chainId: 'x', prompt: 'y' });
} catch (error) {
  if (error instanceof RateLimitError) {
    console.log(`Retry after ${error.retryAfterMs}ms`);
  } else if (error instanceof QuotaExceededError) {
    console.log(`Quota exceeded: ${error.used}/${error.limit} (${error.tier})`);
  } else if (error instanceof TimeoutError) {
    console.log(`Timed out after ${error.timeoutMs}ms`);
  } else if (error instanceof AuthenticationError) {
    console.log('Invalid API key');
  } else if (error instanceof WorkerError) {
    console.log('Worker failed - will retry');
  } else if (error instanceof FileProcessingError) {
    console.log(`File error: ${error.reason} - ${error.filename}`);
  } else if (isRetryableError(error)) {
    console.log('Transient error - safe to retry');
  }
}

Pricing

BYOK (Bring Your Own Key) — You pay AI providers directly. Hydra charges only for API access:

| Tier | Price/mo | Requests | Overage | |------|----------|----------|---------| | Free | $0 | 1,000 | Blocked | | Starter | $9 | 25,000 | $0.50/1K | | Pro | $29 | 100,000 | $0.40/1K | | Scale | $79 | 500,000 | $0.30/1K |

TypeScript Support

Full type safety with comprehensive types:

import type {
  GenerateOptions,
  GenerateResult,
  StreamCallbacks,
  FileAttachment,
  ChainInfo,
  ModelInfo,
} from 'hydra-aidirector';

Requirements

• Node.js 18+
• TypeScript 5+ (optional but recommended)

License

MIT © Hydra