freelm — free, always-up LLM client for Node.js & TypeScript

freelm is a free, always-up LLM client for Node.js/TypeScript that pools multiple free-tier LLM providers — OpenRouter, Google Gemini (AI Studio), NVIDIA NIM, Groq, Cerebras, and Mistral — behind one OpenAI-compatible call (with streaming), with automatic key rotation, cross-provider failover, circuit breaking, and live free-model discovery. Drop in whichever free keys you have and your app keeps talking to an LLM even when one source rate-limits or goes down.

The TypeScript port of freelm for Python — same API, same behavior. Zero runtime dependencies (uses the built-in fetch).

Install

npm install freelm

Quick start

import { FreeLLM } from "freelm";

const llm = FreeLLM.fromEnv();                 // reads provider keys from env
console.log(await llm.text("Explain black holes in one sentence."));

Explicit config:

import { FreeLLM, OpenRouter, GoogleAIStudio, NIM, Groq, Cerebras, Mistral } from "freelm";

const llm = new FreeLLM(
  [
    new OpenRouter("sk-or-..."),
    new GoogleAIStudio("AIza..."),
    new Groq("gsk_..."),
    new Cerebras("csk-..."),
    new Mistral("..."),
    new NIM("nvapi-..."),
  ],
  { strategy: "quota_aware" },                  // priority | round_robin | quota_aware | latency
);

const r = await llm.chat([{ role: "user", content: "Write a haiku about failover." }], { model: "chat:fast" });
console.log(r.text, "via", r.provider);

Streaming

for await (const chunk of llm.stream("Stream me some tokens")) {
  process.stdout.write(chunk);
}

Streaming fails over between providers before the first token; once tokens flow it stays on that provider.

Drop-in OpenAI shim

// import OpenAI from "openai";
import { OpenAI } from "freelm/compat";

const client = new OpenAI();                    // backed by FreeLLM.fromEnv()
const r = await client.chat.completions.create({
  model: "auto",
  messages: [{ role: "user", content: "hi" }],
});
console.log(r.choices[0].message.content);

OpenAI-SDK constructor options ({ apiKey, baseURL, ... }) are accepted and ignored — keys come from the environment. stream: true yields chat.completion.chunk-shaped objects:

const stream = await client.chat.completions.create({ model: "auto", messages, stream: true });
for await (const chunk of stream) process.stdout.write(chunk.choices[0].delta.content ?? "");

Model & provider priority

// 1. ModelSpec priority — order a static list (lower = first)
new OpenRouter("sk-or-...", { discover: false, models: [
  modelSpec("meta-llama/llama-3.3-70b-instruct:free", ["chat", "large"], 131072, true, 0),
  modelSpec("openai/gpt-oss-120b:free", ["chat", "large"], 131072, true, 1),
]});

// 2. prefer — bias *discovered* lists (exact id, else substring; survives refresh)
new OpenRouter("sk-or-...", { prefer: ["qwen3", "gpt-oss"] });

// 3. per-call ordered fallback chain (ids + aliases mix)
await llm.chat(msgs, { model: ["llama-3.3-70b-versatile", "chat:fast"] });

Provider priority (lower = first) breaks ties in every strategy.

Free-only guard

OpenRouter mixes paid and free models, so it ships with freeOnly: true: a non-:free model id throws ConfigError instead of silently billing you. Opt out: new OpenRouter(key, { freeOnly: false }). Other providers' free-tier accounts are free for every model.

Tool calling, observability, persistence, CLI

// tools / JSON output pass straight through
const r = await llm.chat(msgs, { model: "chat:tools", tools, tool_choice: "auto" });
r.toolCalls;

// watch every attempt/failover/success (keys always masked)
const llm = new FreeLLM(provs, { onEvent: (e) => console.log(e.kind, e.provider, e.model, e.status) });

// carry quota/cooldowns/disabled keys across restarts (~/.cache/freelm/state.json)
new FreeLLM(provs, { persist: true });   // or env FREELM_PERSIST=1

npx freelm chat "explain failover in one line" --stream
npx freelm models --provider openrouter
npx freelm health

Environment variables

| Provider | Key vars (first match wins) | Tier var | |----------|------------------------------|----------| | OpenRouter | OPENROUTER_API_KEY / FREELM_OPENROUTER_KEYS | FREELM_OPENROUTER_TIER | | Google AI Studio | GEMINI_API_KEY / GOOGLE_API_KEY / FREELM_GOOGLE_KEYS | FREELM_GOOGLE_TIER | | NVIDIA NIM | NVIDIA_API_KEY / NIM_API_KEY / FREELM_NIM_KEYS | FREELM_NIM_TIER | | Groq | GROQ_API_KEY / FREELM_GROQ_KEYS | FREELM_GROQ_TIER | | Cerebras | CEREBRAS_API_KEY / FREELM_CEREBRAS_KEYS | FREELM_CEREBRAS_TIER | | Mistral | MISTRAL_API_KEY / FREELM_MISTRAL_KEYS | FREELM_MISTRAL_TIER |

Comma-separate to supply multiple keys per provider.

Virtual models & discovery

Ask by intent — "auto", "chat:fast", "chat:large" — and freelm resolves each to a concrete model per provider. Free model ids churn, so freelm discovers them live from each provider's /models endpoint and caches them. List current free models:

import { listFreeModels } from "freelm";
for (const m of (await listFreeModels()).slice(0, 5)) console.log(m.id, m.tags);

How "always-up" works

• Key pool per provider, rotated to spread load.
• Failover interleaved across providers, so every provider is reached fast.
• Circuit breaker per key — opens after repeated failures, half-opens after a cooldown.
• Retry classification: 429 → cool the key & rotate; 5xx/timeout → backoff; 401/402 → disable the key; model errors → next model.
• Quota guard: per-key requests/minute + requests/day, skipping keys predicted exhausted.

Inspect live state with llm.health().

FAQ

How do I use free LLMs in Node.js or TypeScript?

npm install freelm (Node ≥ 18, zero runtime dependencies), set one or more free API keys (OpenRouter, Google AI Studio, NVIDIA NIM, Groq, Cerebras, or Mistral) as environment variables, and call await FreeLLM.fromEnv().text("..."). freelm picks an available free model and handles rate limits and failover automatically.

Is there an OpenAI-compatible free LLM client for JavaScript?

Yes — import { OpenAI } from "freelm/compat" is a drop-in for the OpenAI SDK (client.chat.completions.create(...), including stream: true), backed by free-tier providers with automatic failover.

How do I avoid free-tier rate limits?

freelm paces each key with a requests-per-minute token bucket plus a daily counter, skips keys predicted to be exhausted, and fails over across providers on 429/402/5xx. Add more keys or providers to raise total throughput.

Is freelm really free?

freelm itself is MIT-licensed. It runs on the providers' own free tiers (verified 2026-06) — actual limits depend on each provider's quota, and you can override them per provider.

License

MIT © Shihab Shahriar Antor / Shahriar Labs. Built by Shihab Shahriar Antor. Python version: pypi.org/project/freelm.