@happyvertical/ai

Unified interface for AI model interactions across multiple providers. Supports OpenAI, LiteLLM, Bifrost, Ollama, Anthropic Claude, Google Gemini, AWS Bedrock, Hugging Face, Claude CLI, Qwen3-TTS, and video-generation providers (Gemini Veo, BytePlus ModelArk/Seedance, Seevio/Seedance 2.5, OpenAI-compatible /v1/videos gateways) with a consistent API for chat, completions, embeddings, streaming, function calling, image operations, asynchronous video generation, text-to-speech, and gateway admin provisioning where available.

Installation

pnpm add @happyvertical/ai

Requires @happyvertical/utils as a peer dependency.

Quick Start

import { getAI } from '@happyvertical/ai';

const ai = await getAI({
  type: 'openai',
  apiKey: process.env.OPENAI_API_KEY!,
  defaultModel: 'gpt-4o'
});

// Chat completion
const response = await ai.chat([
  { role: 'system', content: 'You are a helpful assistant.' },
  { role: 'user', content: 'What is TypeScript?' }
]);
console.log(response.content);

// Simple message (convenience wrapper around chat)
const reply = await ai.message('Explain generics in one sentence');

// Streaming
for await (const chunk of ai.stream([
  { role: 'user', content: 'Write a haiku' }
])) {
  process.stdout.write(chunk);
}

Providers

// OpenAI (default when type is omitted)
const openai = await getAI({ apiKey: 'sk-...' });

// LiteLLM (OpenAI-compatible gateway)
const litellm = await getAI({
  type: 'litellm',
  apiKey: process.env.LITELLM_API_KEY!,
  baseUrl: process.env.LITELLM_BASE_URL || 'https://llm.happyvertical.com/v1',
  defaultModel: process.env.LITELLM_MODEL, // Use a model id returned by /v1/models
});

// Bifrost (OpenAI-compatible gateway with governance admin APIs)
const bifrost = await getAI({
  type: 'bifrost',
  apiKey: process.env.BIFROST_API_KEY!,
  adminUser: process.env.BIFROST_ADMIN_USER,
  adminPassword: process.env.BIFROST_ADMIN_PASSWORD,
  adminUrl: process.env.BIFROST_ADMIN_URL,
  baseUrl: process.env.BIFROST_BASE_URL || 'http://localhost:8080',
  defaultModel: process.env.BIFROST_MODEL,
});

// Ollama (local by default)
const ollama = await getAI({
  type: 'ollama',
  baseUrl: process.env.OLLAMA_BASE_URL || process.env.OLLAMA_HOST || 'http://localhost:11434',
  apiKey: process.env.OLLAMA_API_KEY, // Optional, only needed for remote/cloud hosts
  defaultModel: process.env.OLLAMA_MODEL, // Optional; otherwise the first compatible local model is selected
});

// Bare host:port values are also accepted and normalized to http://
const ollamaNode = await getAI({
  type: 'ollama',
  baseUrl: 'warthog:11434',
});

// Anthropic Claude
const claude = await getAI({ type: 'anthropic', apiKey: process.env.ANTHROPIC_API_KEY! });

// Google Gemini
const gemini = await getAI({ type: 'gemini', apiKey: process.env.GEMINI_API_KEY! });

// AWS Bedrock
const bedrock = await getAI({
  type: 'bedrock',
  region: 'us-east-1',
  credentials: { accessKeyId: '...', secretAccessKey: '...' }
});

// Hugging Face
const hf = await getAI({ type: 'huggingface', apiToken: process.env.HF_TOKEN! });

// Claude CLI (uses Claude Max subscription, no API key needed)
const cli = await getAI({ type: 'claude-cli', defaultModel: 'sonnet' });

// Qwen3-TTS (text-to-speech only)
const tts = await getAI({ type: 'qwen3-tts', endpoint: 'http://localhost:8880' });

// BytePlus ModelArk / Seedance (video generation only)
const modelark = await getAI({
  type: 'byteplus-modelark',
  apiKey: process.env.MODELARK_API_KEY!, // or ARK_API_KEY
});

// Seevio / Seedance 2.5 (video generation only)
const seevio = await getAI({
  type: 'seevio',
  apiKey: process.env.SEEVIO_API_KEY!,
  // Default model is pinned to seedance-2-5.
});

// OpenAI-compatible /v1/videos gateway (video generation only)
const video = await getAI({
  type: 'openai-compat-video',
  baseUrl: process.env.OPENAI_COMPAT_VIDEO_BASE_URL || 'https://llm.happyvertical.com/v1',
  apiKey: process.env.OPENAI_COMPAT_VIDEO_API_KEY!,
});

Video Generation

Video generation is asynchronous: submitVideoGenerationJob returns a JSON-serializable handle immediately, and the render itself runs as a provider-side job that you poll with getVideoGenerationJob. Persist the handle (it has no closures or client instances) to resume polling after a restart, and always call cancelVideoGenerationJob on abort so you are not billed for orphaned renders.

const ai = await getAI({ type: 'gemini', apiKey: process.env.GEMINI_API_KEY! });

const job = await ai.submitVideoGenerationJob({
  prompt: 'A drone shot flying over a coastal cliff at sunrise',
  durationSeconds: 8,
  resolution: '1080p',
  aspectRatio: '16:9',
});

// Persist `job` (plain JSON) and resume polling later, even after a process restart.
let status = await ai.getVideoGenerationJob(job);
while (status.status === 'queued' || status.status === 'running') {
  await new Promise((resolve) => setTimeout(resolve, 5000));
  status = await ai.getVideoGenerationJob(job);
}

if (status.status === 'succeeded') {
  const result = await ai.fetchVideoGenerationResult(job);
  if (result.data) fs.writeFileSync('output.mp4', result.data);
}

// On abort/step cancellation (best-effort — see below):
await ai.cancelVideoGenerationJob(job).catch((error) => {
  console.warn('Could not cancel video job', job.jobId, error);
});

Supported providers: gemini (Veo, via @google/genai long-running operations), byteplus-modelark (Seedance, raw-HTTP ModelArk task API), seevio (native Seedance 2.5 task API), and openai-compat-video (thin adapter over a /v1/videos-shaped REST surface — LiteLLM's Veo passthrough, Sora-shaped gateways). Providers without video support throw NOT_IMPLEMENTED, and ai.getCapabilities() reports videoGeneration: false for them.

Cancellation is best-effort, not a guarantee. Gemini has no cancel endpoint for video-generation operations at all and always throws; ModelArk can only cancel a task that hasn't started rendering yet (queued) and rejects cancellation of a running one. Callers must catch and tolerate a cancelVideoGenerationJob failure rather than treating it as fatal.

byteplus-modelark locally throttles submitVideoGenerationJob to approximate Seedance's documented account limits (QPS 2, small burst / 3 concurrent submissions — pass rateLimit: { requestsPerMinute, maxConcurrent } to override the defaults). This limiter is shared across provider instances constructed with the same apiKey, since a fresh instance is often constructed per pipeline step. It bounds concurrent submit HTTP requests, not the lifetime of the render tasks those submissions create — ModelArk doesn't expose a way to observe when a task actually finishes rendering, so task-lifetime concurrency isn't tracked. submitVideoGenerationJob is also registered with the shared rate-limit pacing wrapper, so rateLimit.enabled works the same way it does for chat.

Every provider exposes a cheap validateVideoGenerationAccess() auth-shaped check (a low-cost call — the exact shape is provider-specific: a model/task list for Gemini/ModelArk, a single-resource probe for openai-compat-video since LiteLLM's list route requires a parameter this generic adapter can't supply). It is not free — callers on a hot path must cache the result themselves rather than calling it on every iteration.

Seevio / Seedance 2.5

seevio is an independent adapter, not a ModelArk compatibility mode. It pins generation to seedance-2-5, which supports 4–30 second 480p/720p video, native audio, adaptive aspect ratio, first/last-frame image-to-video, and up to 50 multimodal reference assets. It rejects floating or alternate model IDs so an application does not silently move to a different billed model.

Use the generic referenceMedia extension for public HTTPS image, video, and audio URLs. Existing referenceImages remains available; Seevio accepts it only when every value is a public HTTPS URL because its API retrieves media directly and has no upload endpoint. Buffers and data URLs are intentionally rejected. One or two image-only references select first/last-frame image-to-video; otherwise media references select reference-to-video.

const job = await seevio.submitVideoGenerationJob({
  prompt: 'Use the product image and camera movement reference',
  durationSeconds: 8,
  resolution: '720p',
  aspectRatio: 'adaptive',
  generateAudio: true,
  referenceMedia: [
    { type: 'image', url: 'https://assets.example.com/product.jpg' },
    { type: 'video', url: 'https://assets.example.com/motion.mp4', durationSeconds: 4 },
  ],
});

Seevio reserves credits on submission. The serialized job handle preserves the reserved credit metadata; status results expose normalized billing data and result URL expiry metadata. Submit transport failures are never retried because Seevio documents no idempotency key. Polling is enforced at no more than once per task every 10 seconds. Seevio does not document task cancellation, so cancelVideoGenerationJob explicitly throws NOT_IMPLEMENTED.

Generated URLs are accepted only from https://cdn.seevio.ai by default (or from resultUrlOrigins you explicitly review); fetchVideoGenerationResult checks every redirect against that allow-list before downloading bytes. The unbilled access check is a random GET /v1/tasks/:id: an authenticated 404 is considered valid access, while 401, 403, 402, and 429 retain distinct errors.

fetchVideoGenerationResult limits downloads to 200 MiB by default (override with maxResultBytes), verifies a video/* response type, and enforces the provider timeout while streaming—before buffering the result. For a Node deployment behind an HTTPS proxy, enable Node's environment-proxy support before startup (for example NODE_USE_ENV_PROXY=1 or node --use-env-proxy on supported Node releases); this adapter uses standard fetch and does not silently configure a proxy dispatcher.

Gateway Admin

Gateway providers that support provisioning expose ai.admin.

const ai = await getAI({
  type: 'bifrost',
  apiKey: process.env.BIFROST_API_KEY!,
  adminUrl: process.env.BIFROST_ADMIN_URL || 'http://localhost:8080',
  adminUser: process.env.BIFROST_ADMIN_USER!,
  adminPassword: process.env.BIFROST_ADMIN_PASSWORD!,
  baseUrl: 'http://localhost:8080',
});

const project = await ai.admin!.createProject({
  name: 'Tenant A Production',
  tenantId: 'customer-tenant-a',
  budget: { maxLimit: 100, resetDuration: '1M' },
});

const key = await ai.admin!.createVirtualKey({
  name: 'Tenant A API Key',
  projectId: project.id,
  providerConfigs: [
    {
      provider: 'openai',
      weight: 1,
      allowedModels: ['gpt-4o-mini'],
    },
  ],
  keyIds: ['*'],
  budget: { maxLimit: 25, resetDuration: '1M' },
  rateLimit: {
    tokenMaxLimit: 10000,
    tokenResetDuration: '1h',
    requestMaxLimit: 100,
    requestResetDuration: '1m',
  },
});

console.log(key.key);

LiteLLM uses the same SDK surface, mapping projects to LiteLLM teams and virtual keys to /key/generate.

Generation Safety

Every generative provider constructor applies the same safe defaults, including instances created directly instead of through getAI():

• 4,096 output tokens per request
• 1,024 reasoning tokens per request
• one generated image per request
• a 120-second provider timeout
• zero provider retries (one total upstream attempt)
• local rejection with AI_LIMIT_EXCEEDED before transport when a ceiling is exceeded

Reasoning is opt-in. The 1,024-token default is a ceiling for explicitly requested reasoning, not an automatically enabled thinking budget.

An approved workload can raise a client-level ceiling explicitly. Prefer a workload-specific client so the wider limit is not shared accidentally.

const ai = await getAI({
  type: 'bifrost',
  apiKey: process.env.BIFROST_API_KEY!,
  baseUrl: process.env.BIFROST_BASE_URL!,
  defaultModel: 'gemini-2.5-flash',
  timeout: 105_000,
  maxRetries: 0,
  generationLimits: {
    maxOutputTokens: 8192,
    maxReasoningTokens: 1024,
  },
  usageTags: {
    app: 'writer',
    environment: 'production',
    feature: 'long-form',
  },
  onRequest: (event) => telemetry.record(event),
});

const controller = new AbortController();
const result = await ai.message('Write the approved long-form response', {
  maxTokens: 8192,
  timeout: 105_000,
  signal: controller.signal,
  reasoning: { effort: 'low', maxTokens: 1024 },
});

signal and the SDK timeout are composed and propagated to the provider. A client abort is useful for correctness, but it does not guarantee that a remote provider stops work or billing. Token ceilings and single attempts are the primary cost controls.

onRequest receives one prompt-free terminal event for success, failure, timeout, caller abort, or local rejection. Events include requested and effective token ceilings plus sanitized usageTags; prompts, responses, and credentials are never included. Legacy thinkingLevel options remain available as deprecated aliases and are normalized through the reasoning ceiling.

Opt-In Rate-Limit Pacing

Use rateLimit when multiple calls share the same provider budget and you want getAI() to serialize requests, honor Retry-After hints, and retry only rate-limit failures.

Pacing is enabled when:

• you set enabled: true, or
• you omit enabled and set any pacing field such as key, cooldownMs, initialDelayMs, or maxAttempts

const ai = await getAI({
  type: 'gemini',
  apiKey: process.env.GEMINI_API_KEY!,
  defaultModel: 'gemini-2.5-flash',
  rateLimit: {
    enabled: true,
    key: 'gemini:shared-batch-key',
    cooldownMs: 2000,
    initialDelayMs: 15000,
    maxAttempts: 2,
  },
});

• key coordinates pacing across multiple clients in the same process
• cooldownMs spaces successful calls that share the same budget
• initialDelayMs is the fallback retry delay when the provider omits Retry-After
• maxAttempts counts the first call plus any rate-limit retries

When rateLimit is omitted, or enabled: false is set explicitly, no in-process retry wrapper is applied. Provider retries still default to zero.

rateLimit Options

| Field | Type | Default | Notes | |------|------|---------|------| | enabled | boolean | unset | Set to true for explicit opt-in, or false to force pacing off even if other pacing fields are present | | key | string | derived | Shared budget key; clients with the same key coordinate with each other | | cooldownMs | number | 0 | Minimum delay after a successful call before the next call with the same key | | initialDelayMs | number | 5000 | Fallback retry delay when the provider does not return Retry-After | | maxAttempts | number | 1 | Total attempts, including the initial call; increase explicitly only for an approved workload | | requestsPerMinute | number | provider-specific | Used by qwen3-tts local token-bucket limiting | | maxConcurrent | number | provider-specific | Used by qwen3-tts local concurrency limiting |

• If key is omitted, @happyvertical/ai derives a provider-scoped key from the configured credentials
• Setting any of key, cooldownMs, initialDelayMs, or maxAttempts also opts in when enabled is omitted
• Only normalized rate-limit failures are retried
• stream() is left unchanged; pacing is applied to the promise-returning request methods

Example quota-sensitive batch workload:

const ai = await getAI({
  type: 'gemini',
  apiKey: process.env.GEMINI_API_KEY!,
  defaultModel: 'gemini-2.5-flash',
  rateLimit: {
    enabled: true,
    key: 'praeco:multi-site-analysis',
    cooldownMs: 2000,
    initialDelayMs: 15000,
    maxAttempts: 2,
  },
});

for (const site of sites) {
  const summary = await ai.message(`Summarize anomalies for ${site.name}`);
  console.log(site.name, summary);
}

Environment Variables

getAI() reads HAVE_AI_* variables. Explicit options passed to getAI() take precedence over those env vars.

| Variable | Purpose | |----------|---------| | HAVE_AI_PROVIDER / HAVE_AI_TYPE | Provider type | | HAVE_AI_MODEL / HAVE_AI_DEFAULT_MODEL | Default model | | HAVE_AI_API_KEY | API key (fallback) | | HAVE_AI_BASE_URL | Custom base URL | | HAVE_AI_TIMEOUT | Request timeout (ms) | | HAVE_AI_MAX_RETRIES | Max retry attempts |

Node Auto-Detection Env Vars

getAIAuto() also checks provider-specific Node.js environment variables:

• LITELLM_BASE_URL, LITELLM_API_KEY
• OLLAMA_HOST, OLLAMA_BASE_URL, OLLAMA_API_KEY
• OPENAI_API_KEY
• ANTHROPIC_API_KEY
• GEMINI_API_KEY, GOOGLE_API_KEY
• HF_TOKEN
• AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_DEFAULT_REGION
• MODELARK_API_KEY / ARK_API_KEY, MODELARK_BASE_URL
• OPENAI_COMPAT_VIDEO_BASE_URL, OPENAI_COMPAT_VIDEO_API_KEY

API Overview

Factory Functions

• getAI(options) — Creates a provider instance by type
• getAIAuto(options) — Auto-detects provider from credentials

AIInterface Methods

All providers implement AIInterface:

| Method | Description | |--------|-------------| | chat(messages, options?) | Chat completion returning AIResponse | | message(text, options?) | Simple single-turn convenience method | | complete(prompt, options?) | Text completion | | stream(messages, options?) | Streaming chat (async iterable) | | embed(text, options?) | Text embeddings | | embedImage(image, options?) | Image embeddings (Gemini and Bedrock native, OpenAI and Ollama via describe-then-embed) | | describeImage(image, prompt?, options?) | Image description via vision models | | generateImage(prompt, options?) | Image generation (DALL-E, Imagen, Titan Image Generator, Ollama-compatible image models) | | submitVideoGenerationJob(options) | Submit an async video-generation job, returning a serializable handle | | getVideoGenerationJob(handle) | Poll job status (queued|running|succeeded|failed|cancelled) | | fetchVideoGenerationResult(handle) | Fetch the generated video once the job has succeeded | | cancelVideoGenerationJob(handle) | Cancel an in-flight video-generation job | | validateVideoGenerationAccess() | Cheap auth-shaped check for video-generation access (cache the result) | | countTokens(text) | Token count estimation | | getModels() | List available models | | getCapabilities() | Query provider capabilities | | synthesizeSpeech(text, options?) | Text-to-speech synthesis | | streamSpeech(text, options?) | Streaming TTS | | cloneVoice(options) | Clone a voice from audio sample | | designVoice(options) | Design a voice via text description | | getVoices(options?) | List available voices |

Error Types

All extend AIError: AuthenticationError, RateLimitError, ModelNotFoundError, ContextLengthError, ContentFilterError.

• AIError.retryable distinguishes retryable failures from terminal ones
• RateLimitError.retryAfter exposes provider retry hints in seconds when available

try {
  await ai.chat(messages);
} catch (error) {
  if (error instanceof RateLimitError && error.retryable) {
    console.log('retry after seconds:', error.retryAfter);
  }
}

Legacy Classes

AIClient, OpenAIClient, AIThread, and AIMessageClass are exported for backward compatibility. New code should use getAI() and the AIInterface methods.

Function Calling

const response = await ai.chat([
  { role: 'user', content: 'What is the weather in Tokyo?' }
], {
  tools: [{
    type: 'function',
    function: {
      name: 'get_weather',
      description: 'Get weather for a location',
      parameters: {
        type: 'object',
        properties: { location: { type: 'string' } },
        required: ['location']
      }
    }
  }]
});

if (response.toolCalls) {
  console.log(response.toolCalls[0].function.name);
}

Usage Tracking

Track token usage, costs, and performance across all providers with the onUsage callback:

const ai = await getAI({
  type: 'openai',
  apiKey: process.env.OPENAI_API_KEY!,
  onUsage: (event) => {
    console.log(`[${event.provider}/${event.model}] ${event.operation}: ${event.usage?.totalTokens} tokens in ${event.duration}ms`);
    // Or: save to database, send to analytics, aggregate in-memory, etc.
  },
});

The UsageEvent payload:

| Field | Type | Description | |-------|------|-------------| | provider | string | Provider name ('openai', 'anthropic', 'gemini', etc.) | | model | string | Model used (e.g. 'gpt-4o', 'claude-3-5-sonnet-20241022') | | operation | string | 'chat' | 'complete' | 'message' | 'embed' | 'stream' | ... | | usage? | TokenUsage | { promptTokens, completionTokens, totalTokens } (if available) | | duration | number | Wall-clock time in milliseconds | | timestamp | Date | When the call completed | | tags? | Record<string, string> | Merged from global + per-call usageTags |

• Works with all providers and methods (chat, complete, message, embed, stream)
• complete() and message() report through their underlying chat() call
• Errors thrown inside onUsage are silently caught and will not affect API results

Tagging Usage Events

Attach custom tags to correlate usage with features, users, or workflows:

// Global tags applied to every call
const ai = await getAI({
  type: 'openai',
  apiKey: process.env.OPENAI_API_KEY!,
  usageTags: { app: 'indagator', team: 'news' },
  onUsage: (event) => {
    console.log(event.tags); // { app: 'indagator', team: 'news', feature: 'summarize' }
  },
});

// Per-call tags merge over global tags
await ai.chat(messages, {
  usageTags: { feature: 'summarize', userId: 'u_123' },
});

Claude Code Context

Install context files for AI-assisted development:

npx have-ai-context

License

MIT