@leelaing/ai-provider

Reusable AI provider abstraction for local OpenAI-compatible servers and OpenAI cloud models, with model selection, health checks, normalized responses, streaming where supported, and provider fallback.

Provider types

OpenAICompatibleProvider

Use this for local or third-party servers that expose the OpenAI-compatible /models and /chat/completions contract, including llama.cpp-compatible and LM Studio-style endpoints.

OpenAIResponsesProvider

Use this for OpenAI cloud text generation through the native /responses endpoint. It translates the package's normalized messages into Responses instructions and input, maps maxTokens to max_output_tokens, and normalizes Responses token usage and text output back into the same CompletionResult shape.

Keeping these providers separate prevents local OpenAI-compatible runtimes from being forced to implement the OpenAI Responses protocol while allowing OpenAI cloud integrations to use the current native API.

Features

• OpenAI-compatible chat completion client
• Native OpenAI Responses client
• Local servers such as LM Studio and llama.cpp-compatible endpoints
• Model discovery through /models
• Provider health checks
• Normalized completion results and token usage
• SSE streaming for OpenAI-compatible chat completions
• Abort and timeout support
• Ordered provider fallback across normalized completion providers
• Zero runtime dependencies

Install

npm install @leelaing/ai-provider

Local / OpenAI-compatible example

import { OpenAICompatibleProvider } from '@leelaing/ai-provider';

const local = new OpenAICompatibleProvider({
  id: 'local',
  baseUrl: 'http://127.0.0.1:1234/v1',
  defaultModel: 'deep',
});

const health = await local.health();
const models = await local.listModels();

const result = await local.complete({
  messages: [{ role: 'user', content: 'Hello' }],
});

console.log(health.ok, models, result.text);

OpenAI Responses example

import { OpenAIResponsesProvider } from '@leelaing/ai-provider';

const openai = new OpenAIResponsesProvider({
  id: 'openai-text',
  baseUrl: 'https://api.openai.com/v1',
  apiKey: process.env.OPENAI_API_KEY,
  defaultModel: process.env.OPENAI_TEXT_MODEL,
});

const result = await openai.complete({
  messages: [
    { role: 'system', content: 'Write clear spoken-language prose.' },
    { role: 'user', content: 'Draft a short welcome speech.' },
  ],
});

console.log(result.text);

Streaming

Streaming is currently exposed by OpenAICompatibleProvider:

for await (const chunk of local.stream({
  messages: [{ role: 'user', content: 'Tell me a story.' }],
})) {
  process.stdout.write(chunk.text);
}

The Responses provider currently exposes normalized non-streaming completion. Responses streaming can be added behind the same package boundary without changing application code that uses complete().

Development

npm install
npm start

Quality checks

npm run check

CI runs the full check suite on supported Node versions for every push to master.

License

MIT