Every LLM has its own SDK. One API for all of them.

Switch between OpenAI, Anthropic, Ollama, Google Gemini, and 38 more — without changing a line of code. provider-kit wraps 42 LLM providers behind one consistent chat() interface, with built-in retry, timeout, error handling, and model fallback routing.

npm install provider-kit

Quickstart

import { createProvider } from 'provider-kit'

const client = await createProvider('openai', process.env.OPENAI_API_KEY)
const stream = client.chatStream('gpt-4o-mini', [
  { role: 'user', content: 'Write a haiku' },
])

for await (const chunk of stream) {
  if (chunk.type === 'content') process.stdout.write(chunk.content)
}

Save as demo.mjs, run node demo.mjs, see the first token in under 60 seconds.

Usage

Basic chat

import { createProvider } from 'provider-kit'

const provider = await createProvider('openai', process.env.OPENAI_API_KEY)
const reply = await provider.chat('gpt-4o-mini', [
  { role: 'user', content: 'Hello' },
])
console.log(reply.content)
// → "Hello! How can I help you today?"

Multiple providers

import { providerRegistry } from 'provider-kit'

await providerRegistry.configure('openai', { apiKey: process.env.OPENAI_API_KEY, defaultModel: 'gpt-4o-mini' })
await providerRegistry.configure('ollama', { baseUrl: 'http://localhost:11434', defaultModel: 'llama3.2' })

const provider = providerRegistry.getProvider('openai')
const reply = await provider.chat('gpt-4o-mini', [{ role: 'user', content: 'Hi' }])

Model auto-routing (health probes + real-time switching)

Periodically detects which models are available and routes to the best one. Changes take effect immediately — no restart needed.

import { createRouter } from 'provider-kit'

const router = createRouter({
  probes: [
    { provider: 'openai',   model: 'gpt-4',        apiKey: process.env.OPENAI_API_KEY },
    { provider: 'openai',   model: 'gpt-4o-mini',  apiKey: process.env.OPENAI_API_KEY },
    { provider: 'anthropic', model: 'claude-3-haiku', apiKey: process.env.ANTHROPIC_API_KEY },
    { provider: 'ollama',   model: 'llama3.2',     baseUrl: 'http://localhost:11434' },
  ],
  strategy: 'latency',          // 'latency' | 'failover' | 'round-robin'
  probeInterval: 30000,          // ping every 30s (0 = manual only)
  onProbeResult: (results) => console.log(results),
})

const reply = await router.chat([{ role: 'user', content: 'Hello' }])
// → auto-routed to the healthiest model

// Manual probe
const status = await router.checkNow()
// → [{ provider, model, ok, latency, error }]

// Stop auto-probe
router.stop()

Streaming

const stream = provider.chatStream('gpt-4o-mini', [
  { role: 'user', content: 'Count to 10' },
])
for await (const chunk of stream) {
  if (chunk.type === 'content') process.stdout.write(chunk.content)
}

Cancel streaming anytime:

import { createCancelSignal } from 'provider-kit'
const { signal, cancel } = createCancelSignal()
setTimeout(() => cancel('User stopped'), 3000)
const stream = provider.chatStream('gpt-4', messages, { signal })

Function Calling (with fallback for non-FC models)

const reply = await provider.chat('gpt-4', messages, {
  tools: [{
    type: 'function',
    function: {
      name: 'get_weather',
      description: 'Get weather for a city',
      parameters: {
        type: 'object',
        properties: { city: { type: 'string' } },
      },
    },
  }],
})
if (reply.toolCalls) {
  console.log(reply.toolCalls[0].name)       // → "get_weather"
  console.log(reply.toolCalls[0].arguments)   // → { city: "Tokyo" }
}

For models without native Function Calling support (e.g. MiniMax, some local models), the provider-kit automatically falls back to text-based ACTION: parsing. If the model outputs:

ACTION: get_weather { "city": "Tokyo" }

It will be parsed into proper toolCalls — no code changes needed.

Streaming with non-FC models

When streaming with a model that doesn't support FC, the stream may return empty content. Call chat() as a fallback:

const stream = provider.chatStream(model, messages, { tools })
for await (const chunk of stream) {
  // non-FC models may yield little or no content here
}
// Fallback: chat() handles both FC and text-based ACTION:
const reply = await provider.chat(model, messages, { tools })

Error handling

Every error is a ProviderError with a consistent structure:

import { ProviderError, withRetry, withTimeout, safeProviderCall } from 'provider-kit'

try {
  const reply = await safeProviderCall(
    () => provider.chat('gpt-4', messages),
    { provider: 'openai', retries: 3, timeout: 30000 }
  )
} catch (e) {
  if (e instanceof ProviderError) {
    console.log(e.provider)      // → "openai"
    console.log(e.statusCode)    // → 429
    console.log(e.retryable)     // → true
    console.log(e.type)          // → "rate_limit" | "auth" | "timeout" | "server_error" | "quota" | "bad_request" | "network" | "unknown"
    console.log(e.message)       // → "Rate limit exceeded — slow down or upgrade your plan"
  }
}

| Type | Meaning | Retryable | |------|---------|-----------| | rate_limit | Too many requests | ✅ | | auth | Bad API key | ❌ | | timeout | Provider didn't respond | ✅ | | server_error | Provider 5xx error | ✅ | | quota | Token budget exhausted | ❌ | | bad_request | Invalid input | ❌ | | network | DNS/connection failure | ✅ | | unknown | Catch-all | depends |

Classify raw errors

import { classifyError } from 'provider-kit'

const err = new Error('429 Too Many Requests')
console.log(classifyError(err, 'openai').type)  // → "rate_limit"

Available providers

| Provider | type | Config | |----------|--------|--------| | OpenAI | openai | apiKey | | Anthropic | anthropic | apiKey | | Google Gemini | gemini | apiKey | | Ollama | ollama | baseUrl (default http://localhost:11434) | | Azure OpenAI | azure | apiKey + baseUrl | | AWS Bedrock | bedrock | AWS credentials | | Cohere | cohere | apiKey | | Any OpenAI-compatible | openai + custom baseUrl | apiKey + baseUrl |

32 more providers available via OpenAI-compatible fallback — just set the baseUrl.

Known Limitations (v1.0.0)

• 10 adapters implemented out of 42 presets. The remaining 32 use OpenAI-compatible fallback. Contributions welcome.
• No TypeScript types. Planned.
• API keys are stored in memory. No OS keychain integration. Do not hardcode keys in source files.

Related

• fairy-guardian — self-healing process cluster for AI model servers

Author: openchat-ai — reach me via GitHub Issues or leave a star ⭐