Fallbakit Node.js SDK

TypeScript SDK for Fallbakit's OpenAI-compatible, local-first chat completions API.

Fallbakit routes to the customer's local Ollama, oMLX, or vLLM runner through the open-source tunnel agent first, then falls back to configured BYOK cloud providers when local inference is unavailable and fallback is allowed.

Install

npm install @fallbakit/sdk

Configuration

export FALLBAKIT_API_KEY=or_your_application_key
export FALLBAKIT_BASE_URL=https://api.fallbakit.com

Keep FALLBAKIT_API_KEY in your environment or secret manager for security, then pass it explicitly when constructing the client. baseURL defaults to https://api.fallbakit.com. For local development, pass baseURL: "http://localhost:8080".

For local examples in this repository:

cp .env.example .env.local

.env.example is set up for the local developer stack:

FALLBAKIT_API_KEY=or_your_generated_api_key
FALLBAKIT_BASE_URL=http://localhost:8080
FALLBAKIT_OPENAI_BASE_URL=http://localhost:8080/v1
FALLBAKIT_MODEL=llama3.2
FALLBAKIT_FALLBACK_PROVIDER=openai
FALLBAKIT_FALLBACK_MODEL=gpt-4o-mini

Basic Chat

import { Fallbakit } from "@fallbakit/sdk";

const apiKey = process.env.FALLBAKIT_API_KEY;
if (!apiKey) throw new Error("FALLBAKIT_API_KEY is required");

const client = new Fallbakit({
  apiKey
});

const response = await client.chat.completions.create({
  model: "llama3.2",
  messages: [{ role: "user", content: "Write a tiny launch checklist." }]
});

console.log(response.choices[0].message.content);

Official OpenAI SDK

Fallbakit also works with the official OpenAI Node.js SDK because the router exposes POST /v1/chat/completions. Use a Fallbakit application API key and include /v1 in the OpenAI SDK base URL.

npm install openai
node examples/openai-sdk-chat.mjs
node examples/openai-sdk-streaming.mjs

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.FALLBAKIT_API_KEY,
  baseURL: process.env.FALLBAKIT_OPENAI_BASE_URL ?? "http://localhost:8080/v1"
});

const completion = await client.chat.completions.create({
  model: process.env.FALLBAKIT_MODEL ?? "llama3.2",
  messages: [{ role: "user", content: "Write a tiny launch checklist." }]
});

Streaming

const stream = await client.chat.completions.create({
  model: "llama3.2",
  messages: [{ role: "user", content: "Stream a short answer." }],
  stream: true
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}

Cloud Fallback Controls

const response = await client.chat.completions.create({
  model: "llama3.2",
  messages: [{ role: "user", content: "Summarize hybrid inference." }],
  fallbackProvider: "openai",
  fallbackModel: "gpt-4o-mini"
});

Supported Fallbakit controls:

• fallback: allow cloud fallback when local cannot serve. Defaults to true.
• forceLocal: require local routing.
• fallbackProvider: request a specific configured BYOK fallback provider.
• fallbackModel: request a specific cloud model for fallback.
• cloudModelOnly: skip local routing and call the selected cloud provider.
• localModelOnly: prevent cloud fallback.
• extraBody: pass additional OpenAI-compatible or Fallbakit-specific request body fields.

You can also pass controls under fallbakit:

await client.chat.completions.create({
  model: "llama3.2",
  messages: [{ role: "user", content: "Hello" }],
  fallbakit: {
    fallbackProvider: "gemini",
    fallbackModel: "gemini/gemini-1.5-flash"
  }
});

Timeouts

const apiKey = process.env.FALLBAKIT_API_KEY;
if (!apiKey) throw new Error("FALLBAKIT_API_KEY is required");

const client = new Fallbakit({
  apiKey,
  timeoutMs: 30_000
});

await client.chat.completions.create({
  model: "llama3.2",
  messages: [{ role: "user", content: "Hello" }],
  timeoutMs: 10_000
});

Per-request timeoutMs overrides the client default.

Errors

API failures throw FallbakitAPIError with:

• status
• code
• response

import { Fallbakit, FallbakitAPIError } from "@fallbakit/sdk";

const apiKey = process.env.FALLBAKIT_API_KEY;
if (!apiKey) throw new Error("FALLBAKIT_API_KEY is required");

try {
  await new Fallbakit({
    apiKey
  }).chat.completions.create({
    model: "llama3.2",
    messages: [{ role: "user", content: "Hello" }]
  });
} catch (error) {
  if (error instanceof FallbakitAPIError) {
    console.error(error.status, error.code, error.message);
  }
}

Development

npm install
npm run typecheck
npm test

Local Verification

Use this flow to confirm the package builds, the tests pass, and the examples can talk to a local Fallbakit router.

1. Start a local model runtime in a separate terminal:

ollama serve
ollama pull llama3.2

2. Start the local Fallbakit stack from the repository root:

cp configs/local-infra.env.example configs/local-infra.env
cp configs/api.env.example configs/api.env
scripts/dev-up.sh
# In the dashboard, create a runner and export its generated RUNNER_* values.
(
  cd open-source/tunnel-agent
  go run ./cmd/fallbakit-agent \
    -api-key="$FALLBAKIT_RUNNER_API_KEY" \
    -runner-id="$FALLBAKIT_RUNNER_ID" \
    -base-url=http://localhost:8080 \
    -local-provider=ollama \
    -local-base-url=http://localhost:11434
)

For vLLM local verification, start vLLM on localhost:8000, create a vLLM runner in the dashboard, then use -local-provider=vllm -local-base-url=http://localhost:8000. Direct OpenAI clients use http://localhost:8000/v1; the Fallbakit agent should use the origin without /v1.

3. In a new terminal, install dependencies and load the example environment:

cd open-source/node-sdk
npm install
npm install --no-save openai
cp .env.example .env.local
set -a
source .env.local
set +a

Replace FALLBAKIT_API_KEY in .env.local with the application API key you created in the dashboard.

4. Run the package checks:

npm run typecheck
npm test

5. Run the local smoke tests against http://localhost:8080:

npm run build
node examples/minimal-chat.mjs
node examples/streaming-chat.mjs
node examples/openai-sdk-chat.mjs
node examples/openai-sdk-streaming.mjs

6. If your local stack has a fallback provider configured, run the fallback example too:

node examples/local-first-with-fallback.mjs

If those commands return model output, the local package setup is working correctly. When you are done, stop the dev stack with scripts/dev-down.sh from the repository root.

Publishing

1. Update version in package.json.
2. Run npm run typecheck and npm test.
3. Publish:

npm publish --access public