llm-mini

Tiny LLM streaming library. <5KB. Zero deps. Framework-agnostic.

Features

• Typed streaming with async iterators
• 4 providers: OpenAI, Anthropic, Google, OpenRouter (1000+ models)
• Structured output with Zod (optional)
• Token counter per request
• <5KB gzipped, zero runtime dependencies
• Framework-agnostic: React, Vue, Svelte, Node, Deno, Bun, Edge

Install

npm install llm-mini

Quick Start

import { streamLLM } from 'llm-mini';

const { stream, response } = streamLLM({
  provider: 'openai',
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'Explain quantum computing in one sentence' }],
});

// Stream tokens to console
for await (const chunk of stream) {
  process.stdout.write(chunk.text);
}

// Get final response with usage stats
const result = await response;
console.log('\nUsage:', result.usage);

Examples

See the examples/ directory for runnable code samples:

• stream-chat.ts - Basic streaming chat
• express-api.ts - Express.js API with SSE streaming
• structured-output.ts - Generate typed objects with Zod

# Run an example
OPENAI_API_KEY=sk-... npx tsx examples/stream-chat.ts

Real-World Example: AI Chatbot with Express

import express from 'express';
import { streamLLM } from 'llm-mini';

const app = express();
app.use(express.json());

app.post('/api/chat', async (req, res) => {
  const { message, provider = 'openai' } = req.body;

  res.setHeader('Content-Type', 'text/event-stream');
  res.setHeader('Cache-Control', 'no-cache');
  res.setHeader('Connection', 'keep-alive');

  const { stream } = streamLLM({
    provider,
    model: provider === 'openrouter' ? 'openai/gpt-4o' : 'gpt-4o',
    messages: [{ role: 'user', content: message }],
  });

  for await (const chunk of stream) {
    res.write(`data: ${JSON.stringify(chunk)}\n\n`);
  }

  res.write('data: [DONE]\n\n');
  res.end();
});

app.listen(3000);

Real-World Example: Next.js API Route

// app/api/chat/route.ts
import { streamLLM } from 'llm-mini';

export async function POST(req: Request) {
  const { messages } = await req.json();

  const { stream } = streamLLM({
    provider: 'anthropic',
    model: 'claude-sonnet-4-20250514',
    messages,
  });

  const encoder = new TextEncoder();
  const readable = new ReadableStream({
    async start(controller) {
      for await (const chunk of stream) {
        controller.enqueue(encoder.encode(chunk.text));
      }
      controller.close();
    },
  });

  return new Response(readable, {
    headers: { 'Content-Type': 'text/plain; charset=utf-8' },
  });
}

API Reference

streamLLM(options)

Returns { stream, response } for streaming and final result.

| Option | Type | Required | Description | |--------|------|----------|-------------| | provider | 'openai' \| 'anthropic' \| 'google' \| 'openrouter' | Yes | LLM provider | | model | string | Yes | Model identifier | | messages | Message[] | Yes | Chat messages | | apiKey | string | No | API key (or env var) | | temperature | number | No | Sampling temperature | | maxTokens | number | No | Max completion tokens | | topP | number | No | Top-p sampling | | stop | string[] | No | Stop sequences | | signal | AbortSignal | No | Abort controller | | headers | Record<string, string> | No | Custom headers | | appUrl | string | No | App URL (OpenRouter rankings) | | appName | string | No | App name (OpenRouter rankings) | | timeout | number | No | Request timeout in ms (default: 30000) | | retry | RetryOptions | No | Retry configuration | | cache | CacheOptions | No | Response caching | | debug | DebugOptions | No | Debug logging |

generateText(options)

One-shot text generation (waits for full response).

import { generateText } from 'llm-mini';

const response = await generateText({
  provider: 'anthropic',
  model: 'claude-sonnet-4-20250514',
  messages: [{ role: 'user', content: 'Hello' }],
});

console.log(response.text);
console.log(response.usage);

generateObject(options)

Structured output with Zod schema validation.

import { generateObject } from 'llm-mini';
import { z } from 'zod';

const result = await generateObject({
  provider: 'openai',
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'Create a user profile' }],
  schema: z.object({
    name: z.string(),
    age: z.number(),
    email: z.string().email(),
  }),
});

console.log(result.data); // Typed as { name: string; age: number; email: string }

Providers

OpenAI

import { streamLLM } from 'llm-mini';

// Using env var OPENAI_API_KEY
const { stream } = streamLLM({
  provider: 'openai',
  model: 'gpt-4o',
  messages: [...],
});

// Or pass key directly
const { stream } = streamLLM({
  provider: 'openai',
  model: 'gpt-4o',
  apiKey: 'sk-...',
  messages: [...],
});

Anthropic

const { stream } = streamLLM({
  provider: 'anthropic',
  model: 'claude-sonnet-4-20250514',
  messages: [...],
});
// Uses env var ANTHROPIC_API_KEY

Google

const { stream } = streamLLM({
  provider: 'google',
  model: 'gemini-3.6-flash',
  messages: [...],
});
// Uses env var GOOGLE_API_KEY

OpenRouter

Access 1000+ models with a single API key.

const { stream } = streamLLM({
  provider: 'openrouter',
  model: 'openai/gpt-4o',        // OpenRouter model format
  messages: [...],
  appUrl: 'https://myapp.com',   // Optional: for rankings
  appName: 'My App',             // Optional: for rankings
});
// Uses env var OPENROUTER_API_KEY

OpenRouter model format: provider/model-name (e.g., anthropic/claude-3-opus, meta-llama/llama-3-70b-instruct)

Advanced Options

Timeout

Set a custom request timeout (default: 30 seconds):

const { stream } = streamLLM({
  provider: 'openai',
  model: 'gpt-4o',
  messages: [...],
  timeout: 10000, // 10 seconds
});

Retry with Exponential Backoff

Automatically retry failed requests with configurable backoff:

const { stream } = streamLLM({
  provider: 'openai',
  model: 'gpt-4o',
  messages: [...],
  retry: {
    maxRetries: 3,        // default: 3
    initialDelay: 1000,   // default: 1000ms
    maxDelay: 10000,      // default: 10000ms
    retryableStatuses: [408, 429, 500, 502, 503, 504], // default
  },
});

Response Caching

Cache responses to avoid duplicate requests:

const { stream } = streamLLM({
  provider: 'openai',
  model: 'gpt-4o',
  messages: [...],
  cache: {
    enabled: true,
    maxSize: 100,      // max entries (default: 100)
    ttl: 300000,       // time-to-live in ms (default: 5 min)
  },
});

Debug Logging

Enable detailed request/response logging:

const { stream } = streamLLM({
  provider: 'openai',
  model: 'gpt-4o',
  messages: [...],
  debug: {
    enabled: true,
    logRequests: true,
    logResponses: true,
    logger: (message, data) => console.log(`[llm-mini] ${message}`, data),
  },
});

Error Handling

import { streamLLM, LLMError } from 'llm-mini';

try {
  const { response } = streamLLM({ ... });
  await response;
} catch (err) {
  if (err instanceof LLMError) {
    console.error(`Provider: ${err.provider}`);
    console.error(`Status: ${err.status}`);
    console.error(`Message: ${err.message}`);
  }
}

Abort Support

const controller = new AbortController();

const { stream } = streamLLM({
  provider: 'openai',
  model: 'gpt-4o',
  messages: [...],
  signal: controller.signal,
});

// Cancel after 5 seconds
setTimeout(() => controller.abort(), 5000);

try {
  for await (const chunk of stream) {
    process.stdout.write(chunk.text);
  }
} catch (err) {
  if (err instanceof LLMError) {
    console.log('Request aborted');
  }
}

Comparison with Vercel AI SDK

| Feature | llm-lite | Vercel AI SDK | |---------|----------|---------------| | Bundle size | <5KB | ~50KB+ | | Dependencies | 0 | Many | | Framework | Any | React-focused | | Providers | 4 (+1000 via OpenRouter) | 100+ | | Streaming | Yes | Yes | | Structured output | Yes (Zod) | Yes (Zod) | | Agent primitives | No | Yes | | UI hooks | No | Yes | | Token counter | Built-in | Via callbacks |

Choose llm-lite if: You want a lightweight, framework-agnostic streaming library without vendor lock-in.

Choose Vercel AI SDK if: You need agent primitives, UI hooks, or 100+ provider support on Vercel.

TypeScript

Full TypeScript support with strict types.

import type { Message, StreamOptions, TokenUsage } from 'llm-mini';

License

MIT