fuse-llm
v0.1.2
Published
A clean, object-oriented TypeScript library for calling any LLM — OpenAI, Claude, Gemini, OpenRouter, HuggingFace
Maintainers
Readme
fuse-llm
A clean, zero-dependency TypeScript library for calling OpenAI, Claude, Gemini, OpenRouter, and HuggingFace — with one unified API.
Why fuse-llm?
- One interface, every provider —
chat()andstream()work the same regardless of which LLM you use - Zero runtime dependencies — built on native
fetch,ReadableStream, andTextDecoder(Node ≥ 18) - Full TypeScript — strict types, model name autocomplete, dual ESM + CJS build
- Streaming built-in — async generators across all providers, including Gemini's EOF-terminated streams
- Predictable errors — every failure throws an
OmniLLMErrorwithstatus,provider, andrawError
Install
npm install fuse-llmRequires Node.js ≥ 18.
Providers
| Class | Provider | Required key |
|---|---|---|
| OpenAI | OpenAI | apiKey |
| Claude | Anthropic Claude | apiKey |
| Gemini | Google Gemini | apiKey |
| OpenRouter | OpenRouter | apiKey |
| HuggingFace | HuggingFace Inference | apiKey |
Quick start
import { OpenAI } from 'fuse-llm';
const client = new OpenAI({
apiKey: 'sk-...',
model: 'gpt-4o',
});
// Non-streaming
const response = await client.chat({
messages: [{ role: 'user', content: 'Hello!' }],
});
console.log(response.text); // "Hello! How can I help you?"
// Streaming
for await (const chunk of client.stream({
messages: [{ role: 'user', content: 'Count to 5.' }],
})) {
process.stdout.write(chunk.text);
}Core concepts
Messages
Every call takes a messages array. Three roles are supported:
{ role: 'system', content: 'You are a helpful assistant.' }
{ role: 'user', content: 'What is 2 + 2?' }
{ role: 'assistant', content: 'The answer is 4.' }System messages are handled correctly per-provider automatically:
- Claude — extracted into the top-level
systemfield (Anthropic's requirement) - Gemini — mapped to
systemInstruction - OpenAI / OpenRouter / HuggingFace — passed as-is
chat() — full response
const response = await client.chat({ messages });
response.text // the model's reply as a string
response.model // exact model name the API used
response.usage // { promptTokens, completionTokens, totalTokens }
response.raw // the full raw JSON body from the providerstream() — incremental chunks
for await (const chunk of client.stream({ messages })) {
chunk.text // the text delta for this chunk
chunk.rawChunk // the raw SSE chunk from the provider
}Constructor options
All providers accept the same base options:
{
apiKey: string, // required — your API key
model: string, // required — the default model
temperature: number, // optional — sampling temperature (0–2)
maxTokens: number, // optional — max tokens to generate
baseUrl: string, // optional — override the API base URL (e.g. a proxy)
timeoutMs: number, // optional — request timeout in ms (default: 30 000)
}Per-call overrides
Any option can be overridden for a single call without changing the client:
const client = new OpenAI({ apiKey: '...', model: 'gpt-4o' });
const response = await client.chat({
messages: [...],
model: 'gpt-4o-mini', // use a different model just this once
temperature: 0.2,
maxTokens: 256,
timeoutMs: 10_000,
});Provider examples
OpenAI
import { OpenAI } from 'fuse-llm';
const client = new OpenAI({ apiKey: 'sk-...', model: 'gpt-4o' });
const response = await client.chat({
messages: [
{ role: 'system', content: 'You are a concise assistant.' },
{ role: 'user', content: 'What is TypeScript?' },
],
});
console.log(response.text);Available models (with autocomplete):
gpt-4o, gpt-4o-mini, gpt-4-turbo, gpt-4, gpt-3.5-turbo, o1, o1-mini, o3, o3-mini
Claude (Anthropic)
import { Claude } from 'fuse-llm';
const client = new Claude({
apiKey: 'sk-ant-...',
model: 'claude-opus-4-5',
maxTokens: 2048, // defaults to 4096 if omitted — Anthropic always requires this
});
const response = await client.chat({
messages: [
{ role: 'system', content: 'Reply only in haiku.' },
{ role: 'user', content: 'Explain recursion.' },
],
});Available models: claude-opus-4-5, claude-sonnet-4-5, claude-haiku-3-5, claude-3-5-sonnet-20241022, claude-3-5-haiku-20241022, and more.
Claude-specific option:
new Claude({
apiKey: '...',
model: 'claude-opus-4-5',
anthropicVersion: '2023-06-01', // defaults to '2023-06-01'
});Gemini (Google)
import { Gemini } from 'fuse-llm';
const client = new Gemini({
apiKey: 'AIza...',
model: 'gemini-2.0-flash',
});
// Stream a response
for await (const chunk of client.stream({
messages: [{ role: 'user', content: 'Write a poem about the Himalayas.' }],
})) {
process.stdout.write(chunk.text);
}Available models: gemini-2.5-pro, gemini-2.5-flash, gemini-2.0-flash, gemini-2.0-flash-lite, gemini-1.5-pro, gemini-1.5-flash
Note: Gemini passes the API key in the URL query string (
?key=...), not in headers — fuse-llm handles this automatically.
OpenRouter
OpenRouter gives you access to hundreds of models (Llama, Mistral, Command R, etc.) through a single API key.
import { OpenRouter } from 'fuse-llm';
const client = new OpenRouter({
apiKey: 'sk-or-...',
model: 'meta-llama/llama-3.1-8b-instruct',
siteUrl: 'https://yourapp.com', // optional — shown in OpenRouter dashboard
siteName: 'My App', // optional — shown in OpenRouter dashboard
});HuggingFace Inference
import { HuggingFace } from 'fuse-llm';
const client = new HuggingFace({
apiKey: 'hf_...',
model: 'meta-llama/Meta-Llama-3.1-8B-Instruct',
});
const response = await client.chat({
messages: [{ role: 'user', content: 'Explain neural networks simply.' }],
});Multi-turn conversations
Pass the full conversation history in messages:
const client = new OpenAI({ apiKey: 'sk-...', model: 'gpt-4o' });
const response = await client.chat({
messages: [
{ role: 'user', content: 'My name is Bishal.' },
{ role: 'assistant', content: 'Hi Bishal! How can I help?' },
{ role: 'user', content: 'What is my name?' },
],
});
console.log(response.text); // "Your name is Bishal."Error handling
Every error is an OmniLLMError — a subclass of Error with extra context:
import { OpenAI, OmniLLMError } from 'fuse-llm';
const client = new OpenAI({ apiKey: 'sk-...', model: 'gpt-4o' });
try {
const response = await client.chat({
messages: [{ role: 'user', content: 'Hello' }],
});
} catch (err) {
if (err instanceof OmniLLMError) {
console.error(err.message); // human-readable description
console.error(err.provider); // 'openai' | 'claude' | 'gemini' | ...
console.error(err.status); // HTTP status code (0 = network error / timeout)
console.error(err.rawError); // full error body from the provider's API
}
}| status | What it means |
|---|---|
| 401 | Bad API key |
| 429 | Rate limited |
| 500 | Provider server error |
| 0 | Network failure or timeout |
Using a proxy / custom base URL
Point any provider at a local proxy or alternative endpoint:
const client = new OpenAI({
apiKey: 'sk-...',
model: 'gpt-4o',
baseUrl: 'http://localhost:8080/v1', // any OpenAI-compatible proxy
});Custom providers
Extend OpenAICompatibleProvider to add your own OpenAI-compatible endpoint:
import { OpenAICompatibleProvider } from 'fuse-llm';
import type { ProviderConfig } from 'fuse-llm';
class MyModel extends OpenAICompatibleProvider {
protected override readonly providerName = 'my-model';
constructor(config: ProviderConfig<string>) {
super(config, {
baseUrl: 'https://my-api.example.com/v1',
});
}
}
const client = new MyModel({ apiKey: 'key-...', model: 'my-model-v1' });Or extend BaseProvider directly for fully custom wire formats (like Claude or Gemini):
import { BaseProvider } from 'fuse-llm';
import type { ChatOptions, OmniResponse, OmniStreamChunk, ProviderConfig } from 'fuse-llm';
class MyCustomProvider extends BaseProvider<string> {
protected readonly providerName = 'my-custom';
constructor(config: ProviderConfig<string>) {
super(config, { baseUrl: 'https://api.example.com' });
}
async chat(options: ChatOptions<string>): Promise<OmniResponse> {
const resolved = this.resolve(options);
// use this.fetchJSON(...) — handles timeout, AbortController, error wrapping
const raw = await this.fetchJSON('...', {}, {}, resolved.timeoutMs);
return { text: '...', model: '...', usage: { promptTokens: 0, completionTokens: 0, totalTokens: 0 }, raw };
}
async *stream(options: ChatOptions<string>): AsyncGenerator<OmniStreamChunk> {
// use this.fetchStream(...) + parseSSEStream from 'fuse-llm/utils/sse'
yield { text: '...', rawChunk: null };
}
}TypeScript types
All public types are exported:
import type {
OmniMessage, // { role, content }
OmniResponse, // { text, usage, model, raw }
OmniStreamChunk, // { text, rawChunk }
TokenUsage, // { promptTokens, completionTokens, totalTokens }
ChatOptions, // options passed to chat() / stream()
ProviderConfig, // constructor options
LLMProvider, // interface both chat() and stream() satisfy
OpenAIModel, // union of known OpenAI model strings
ClaudeModel, // union of known Claude model strings
GeminiModel, // union of known Gemini model strings
} from 'fuse-llm';License
MIT © bishal
