@founder-os/llm-router

Unified chat() interface over Anthropic, OpenAI, and Google Gemini. v0.1 scaffold: provider-agnostic request/response shape, exponential-backoff retries, and a pluggable cost-telemetry hook.

Install

import { chat, type ChatRequest, type ChatResponse } from '@founder-os/llm-router';

Usage

const res = await chat({
  provider: 'anthropic',
  model: 'claude-opus-4-5',
  messages: [
    { role: 'system', content: 'You are concise.' },
    { role: 'user', content: 'What is 2 + 2?' },
  ],
  maxTokens: 256,
});

console.log(res.content);
console.log(res.usage); // { inputTokens, outputTokens }

Per-call cost recorder

Inject a recorder so each feature can be billed separately. Defaults to a no-op.

import { chat } from '@founder-os/llm-router';
import type { CostRecorder } from '@founder-os/llm-router/cost';

const recorder: CostRecorder = {
  async record(event) {
    await db.costEvents.insert(event);
  },
};

await chat({
  provider: 'openai',
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'hi' }],
  orgId: 'org_123',
  feature: 'strategy-copilot',
  costRecorder: recorder,
});

Retries

429s and 5xx are retried up to 3 times with jittered exponential backoff. All other errors fail fast. Override per call:

await chat({
  provider: 'google',
  model: 'gemini-1.5-pro',
  messages: [...],
  retry: { maxRetries: 5, baseDelayMs: 500 },
});

Deferred to v1

Search for TODO(llm-router-v1): markers to locate injection points.

• Per-org budget caps (DB-backed cost recorder enforcement) — src/cost.ts
• Model-tier fallback (cheap-model → expensive-model on quality signal) — src/index.ts
• Response cache (prompt-hash keyed) — src/index.ts
• Streaming responses (chatStream()) — src/index.ts