weysabi

AI orchestration for fullstack devs. Provider failover, structured output, streaming, RAG, guardrails, and prompt management — one dependency, zero markup.

import { createWeysabi } from "weysabi";
import { z } from "zod";

const weysabi = createWeysabi({
  groq: { apiKey: process.env.GROQ_API_KEY },
  openai: { apiKey: process.env.OPENAI_API_KEY },
});

const res = await weysabi.complete({
  model: "groq/llama-4-scout",
  messages: [{ role: "user", content: "Extract the invoice total." }],
  fallbacks: ["openai/gpt-4o-mini"],
  response: { schema: z.object({ total: z.number() }) },
  rag: ["invoices/*.pdf"],
  guardrails: ["pii"],
});

console.log(res.parsed.total, res.latencyMs, res.requestId);

Why Weysabi?

• Your keys, your providers. No gateway, no token markup, no middleman.
• One dependency. Not LangChain + provider SDKs + vector DB. Just weysabi.
• Built-in everything. Structured output, RAG, guardrails, prompts, streaming, caching — all ship in the box.
• Works offline-first. SQLite for conversations and RAG. No cloud dependency.
• No lock-in. Stop paying, the library still works. Your data stays with you.

Features

| Feature | Status | | --------------------------------------------------------------------------------------------------------------- | ------ | | Provider abstraction (OpenAI, Groq, Anthropic, Google, Mistral, DeepSeek, OpenRouter, Together, Nvidia, Ollama) | ✅ | | Custom / OpenAI-compatible endpoints | ✅ | | Custom provider handlers (register your own) | ✅ | | Provider failover (primary → fallbacks) | ✅ | | Circuit breaker + retry + backoff | ✅ | | Streaming (SSE, async iterable) | ✅ | | Structured output (Zod schemas with auto-retry) | ✅ | | Tool calling (auto-execute + chaining) | ✅ | | Prompt templates with {variable} substitution | ✅ | | Prompt registry (register, render, run) | ✅ | | RAG engine (ingest, embed, HNSW search, persist) | ✅ | | Multi-project RAG manager | ✅ | | Guardrails (PII redaction, injection detection, content safety, token limits) | ✅ | | Conversation memory (SQLite/Postgres, auto-truncation) | ✅ | | ChatSDK (prepare + call + record in one) | ✅ | | Framework adapters (Hono, Next.js, Express, Fastify, Elysia) | ✅ | | Caching (InMemory with LRU, Redis, BYO) | ✅ | | Plugin system (lifecycle hooks) | ✅ | | OpenTelemetry integration | ✅ | | Vercel AI SDK adapter | ✅ | | Cost estimation (per-response estimatedCostUsd) | ✅ | | requestId tracking (auto-generated, all logs) | ✅ | | Error status codes (programmatic retry vs. fatal) | ✅ | | WebSocket client (real-time streaming) | ✅ | | CLI (weysabi init, create, complete, stream, config, prompt) | ✅ | | Control plane — projects, conversations, runs, API keys | 🔜 | | Weysabi Cloud — hosted control plane, evals, monitoring | 🔜 |

Install

bun add weysabi

Requires Bun ≥ 1.3.

Providers

Configure any provider with an API key. All providers share the same interface.

const weysabi = createWeysabi({
  openai: { apiKey: process.env.OPENAI_API_KEY },
  anthropic: { apiKey: process.env.ANTHROPIC_API_KEY },
  google: { apiKey: process.env.GOOGLE_API_KEY },
  groq: { apiKey: process.env.GROQ_API_KEY },
  deepseek: { apiKey: process.env.DEEPSEEK_API_KEY },
  mistral: { apiKey: process.env.MISTRAL_API_KEY },
  nvidia: { apiKey: process.env.NVIDIA_API_KEY },
  openrouter: { apiKey: process.env.OPENROUTER_API_KEY },
  together: { apiKey: process.env.TOGETHER_API_KEY },
  ollama: { baseUrl: "http://localhost:11434" },
  // Custom OpenAI-compatible endpoint
  myproxy: { apiKey: "sk-...", baseUrl: "https://my-proxy.com/v1" },
});

Reference models with provider/model-id notation:

weysabi.complete({ model: "groq/llama-4-scout", ... });
weysabi.complete({ model: "openai/gpt-4o", ... });
weysabi.complete({ model: "anthropic/claude-3-5-sonnet-20241022", ... });

Custom Provider Handlers

For providers with non-OpenAI-compatible APIs, register a custom handler:

import { registerProvider } from "weysabi";
import type { ProviderHandler } from "weysabi";

const myHandler: ProviderHandler = {
  buildUrl(baseUrl, modelId, stream) {
    /* ... */
  },
  buildHeaders(apiKey) {
    /* ... */
  },
  buildBody(modelId, messages, opts) {
    /* ... */
  },
  parseResponse(data) {
    /* ... */
  },
  parseStreamChunk(data) {
    /* ... */
  },
};

registerProvider("my-custom-provider", myHandler);

const weysabi = createWeysabi({
  "my-custom-provider": { apiKey: process.env.MY_KEY },
});

Provider Failover

Automatic fallback when a provider fails. Circuit breaker prevents hammering a failing endpoint.

const res = await weysabi.complete({
  model: "groq/llama-4-scout",
  fallbacks: ["openai/gpt-4o-mini", "anthropic/claude-3-5-haiku-20241022"],
  messages: [{ role: "user", content: "Hello" }],
});
// Groq fails → OpenAI fallback → response delivered

Structured Output

Pass a Zod schema. Weysabi validates the response and retries on parse failure.

import { z } from "zod";

const CalendarSchema = z.object({
  events: z.array(
    z.object({
      title: z.string(),
      date: z.string(),
      attendees: z.array(z.string()),
    })
  ),
});

const res = await weysabi.complete({
  model: "groq/llama-4-scout",
  messages: [{ role: "user", content: "Schedule a meeting for next Tuesday." }],
  schema: CalendarSchema,
});

console.log(res.parsed.events);
// Typed as { title: string; date: string; attendees: string[] }[]

On parse failure, throws SchemaValidationError with .raw (raw response) and .issues (Zod errors). Each error carries a statusCode property for programmatic handling.

Streaming

Works with all providers and includes failover — if the primary provider fails mid-stream, the library transparently retries the full request against fallbacks.

const stream = weysabi.stream({
  model: "groq/llama-4-scout",
  messages: [{ role: "user", content: "Write a poem." }],
});

for await (const chunk of stream) {
  if (chunk.content) process.stdout.write(chunk.content);
  if (chunk.usage) console.log("\nTokens:", chunk.usage.totalTokens);
}

Framework Adapters

// Hono, Next.js, Elysia — any Web Fetch framework
import { toResponse } from "weysabi/hono";
// import { toResponse } from "weysabi/next";
// import { toResponse } from "weysabi/elysia";

app.post("/chat", async (c) => {
  const stream = weysabi.stream({ ... });
  return toResponse(stream); // SSE response
});

// Express
import { pipe } from "weysabi/express";
app.post("/chat", async (req, res) => {
  const stream = weysabi.stream({ ... });
  await pipe(stream, res);
});

// Fastify
import { pipe } from "weysabi/fastify";
app.post("/chat", async (req, reply) => {
  const stream = weysabi.stream({ ... });
  await pipe(stream, reply);
});

Batch Requests

Send multiple completions concurrently with a configurable concurrency limit.

const results = await weysabi.batch(
  [
    { model: "groq/llama-4-scout", messages: [{ role: "user", content: "Q1" }] },
    { model: "groq/llama-4-scout", messages: [{ role: "user", content: "Q2" }] },
    { model: "groq/llama-4-scout", messages: [{ role: "user", content: "Q3" }] },
  ],
  { concurrency: 5 }
);

for (const result of results) {
  if (result.ok) {
    console.log(result.data.content);
  } else {
    console.error("Failed:", result.error);
  }
}

Results are returned in request order. Errors are isolated per request.

Prompt Templates

Define, register, and run templates with {variable} substitution.

weysabi.prompts.register({
  id: "classify",
  model: "groq/llama-4-scout",
  messages: [
    {
      role: "system",
      content: "You are a support ticket classifier.",
    },
    {
      role: "user",
      content: `Classify this ticket: {ticket_text}

Categories: billing, technical, account, feature_request

Respond with just the category.`,
    },
  ],
});

// Render + complete in one call
const result = await weysabi.prompts.run("classify", {
  ticket_text: "I was overcharged for my subscription",
});

// Or render separately for inspection
const messages = weysabi.prompts.render("classify", {
  ticket_text: "Login is broken",
});

RAG (Retrieval-Augmented Generation)

Built-in vector search without an external vector database. Ingest documents, auto-chunk, embed, and search.

import { RagEngine } from "weysabi/rag";

const rag = new RagEngine({
  dbPath: ".weysabi/knowledge.db",
  embeddingModel: "openai/text-embedding-3-small",
});

rag.setProviders(
  { provider: "openai", apiKey: process.env.OPENAI_API_KEY },
  { openai: { apiKey: process.env.OPENAI_API_KEY } }
);

// Ingest files
await rag.load("docs/manual.pdf", "docs/faq.md");

// Query
const results = await rag.query("How do I reset my password?");
console.log(results[0].content);

Multi-project management:

import { RagManager } from "weysabi/rag";

const manager = new RagManager({
  basePath: ".weysabi/rag",
  providers: { embeddingProvider: { provider: "openai", apiKey: "..." } },
});

const docs = manager.project("docs-v2");
await docs.load("guides/");

const hits = await docs.query("pricing");

Guardrails

PII redaction, prompt injection detection, content moderation, and output token limits.

const res = await weysabi.complete({
  model: "groq/llama-4-scout",
  messages: [{ role: "user", content: "My email is [email protected]" }],
  guardrails: ["pii"], // Redact emails, phones, SSNs
  // guardrails: ["injection"],            // Detect prompt injection
  // guardrails: ["content"],             // Moderate toxic content
  // guardrails: [{ type: "limits", maxOutputTokens: 100 }],
});

Guardrails can also be used standalone:

import { guardrail } from "weysabi/guardrails";

const result = await guardrail("pii", "My SSN is 123-45-6789");
console.log(result.redacted); // "My SSN is [REDACTED]"

Custom Guardrails

weysabi.guardrail("my-check", {
  scope: "output",
  validate: (text) => text.length < 1000,
  onViolation: (match) => console.log("Blocked:", match),
});

Conversation Memory

Persistent chat history with automatic context management. SQLite by default, Postgres for production.

import { ConversationMemory } from "weysabi/chat";

const memory = new ConversationMemory({
  dbPath: ".weysabi/chat.db",
  maxHistoryTokens: 16384,
});

// Prepare context — no API call
const ctx = memory.prepare("user-abc", {
  message: "Hi, my name is Bob",
  system: "You are a helpful assistant",
});

// Call your provider SDK natively
const response = await anthropic.messages.create({
  model: "claude-3-5-sonnet-20241022",
  system: ctx.system,
  messages: ctx.messages,
});

// Record the turn
memory.record("user-abc", {
  userMessage: { content: "Hi, my name is Bob" },
  assistantMessage: { content: response.content[0].text },
});

// Next turn — history is loaded automatically
const ctx2 = memory.prepare("user-abc", {
  message: "What's my name?",
});
// ctx2.messages includes full history

Postgres for production:

import postgres from "postgres";
import { ConversationMemory, PgSessionStore } from "weysabi/chat";

const sql = postgres("postgres://user:pass@host:5432/db");
const memory = new ConversationMemory({
  store: new PgSessionStore(sql),
});

Tool Calling

Define tools with Zod schemas. The library handles execution and chaining.

const res = await weysabi.complete({
  model: "groq/llama-4-scout",
  messages: [{ role: "user", content: "What's the weather in London?" }],
  tools: [
    {
      name: "get_weather",
      description: "Get current weather",
      schema: z.object({ city: z.string() }),
      execute: async ({ city }) => fetchWeather(city),
    },
  ],
});

Caching

import { InMemoryCache } from "weysabi/cache";

const weysabi = createWeysabi(providers, {
  cache: new InMemoryCache(60_000), // 60s TTL
});

InMemoryCache supports max-size LRU eviction — pass a second argument to prevent unbounded growth:

const cache = new InMemoryCache(60_000, 1000); // 1000 entries max

requestId Tracking

Every request gets a unique requestId, auto-generated via crypto.getRandomValues(). Pass your own for correlation:

const res = await weysabi.complete({
  model: "groq/llama-4-scout",
  messages: [{ role: "user", content: "Hello" }],
  requestId: "my-correlation-id",
});

console.log(res.requestId); // "my-correlation-id"

The requestId flows through all structured log entries, making debugging across services straightforward.

Error Handling

Every error class carries a statusCode for programmatic retry vs. fatal distinction:

| Error | statusCode | Meaning | | ---------------------------- | ---------- | ---------------------- | | SchemaValidationError | 422 | Invalid user input | | ProviderNotConfiguredError | 400 | Provider not in config | | PromptNotFoundError | 404 | Template missing | | MissingPromptInputError | 400 | Missing variable | | MaxToolCallsExceededError | 400 | Tool loop limit | | CircuitBreakerOpenError | 503 | Provider recovering | | AllModelsFailedError | 502 | All providers failed | | ProviderRequestError | varies | Upstream HTTP code | | WeysabiError (base) | 500 | Unknown error |

try {
  const res = await weysabi.complete({ ... });
} catch (err) {
  if (err instanceof WeysabiError) {
    if (err.statusCode < 500) {
      console.log("Client error, fix the request:", err.message);
    } else {
      console.log("Server error, can retry:", err.statusCode);
    }
  }
}

Plugins

Lifecycle hooks for telemetry, logging, or custom behavior.

weysabi.use({
  name: "logger",
  onCompleteRequest(req) {
    console.log("Request:", req.model);
    return req;
  },
  onCompleteResponse(res, req) {
    console.log("Response:", res.latencyMs, "ms");
    return res;
  },
  onError(err, { request }) {
    console.error("Failed:", err.message);
  },
});

OpenTelemetry

import { createOtelPlugin } from "weysabi/otel";

weysabi.use(createOtelPlugin({ tracer: trace.getTracer("my-app") }));

Vercel AI SDK

import { createWeysabiProvider } from "weysabi/ai-sdk";

const provider = createWeysabiProvider(weysabi);
const result = await generateText({
  model: provider.languageModel("groq/llama-4-scout"),
  prompt: "Hello",
});

Control Plane Client

Built-in HTTP + WebSocket client for the Weysabi server control plane — projects, conversations, runs, prompts, documents, and API keys.

import { createWeysabiClient, createWebSocketClient } from "weysabi";

// HTTP client
const client = createWeysabiClient({
  baseUrl: "http://localhost:3000",
  apiKey: "sk-admin",
});

const project = client.project("project-1");
const runs = await project.runs.list({ status: "success" });

// WebSocket streaming
const ws = createWebSocketClient({
  baseUrl: "http://localhost:3000",
  apiKey: "sk-admin",
});

const events = ws.stream("POST", "/v1/projects/p/conversations/c/messages/stream", {
  content: "Hello",
  model: "groq/llama-4-scout",
});

CLI

# Scaffold a new project
bunx create-weysabi-app my-app

# Interactive project setup
bun weysabi init

# Test provider connectivity
bun weysabi config validate

# Quick completions
bun weysabi complete "Hello" -m groq/llama-4-scout
bun weysabi stream "Tell me a story" -m openai/gpt-4o-mini

# Manage prompts
bun weysabi prompt list
bun weysabi prompt add classify -f prompts/classify.txt

HTTP Server

Deploy as an OpenAI-compatible API with auth, rate limits, quotas, usage tracking, and admin monitoring.

bun weysabi server --port 3000

Or embed in your app:

import { createServer } from "weysabi-server";

const server = await createServer(weysabi, {
  apiKey: "sk-my-key",
  adminApiKey: "sk-admin-secret",
});

See weysabi-server for full documentation.

Sub-path Exports

import { createWeysabi } from "weysabi";
import { WeysabiError } from "weysabi/errors";
import { toResponse } from "weysabi/sse";
import { pipe } from "weysabi/express";
import { InMemoryCache } from "weysabi/cache";
import { createOtelPlugin } from "weysabi/otel";
import { createWeysabiProvider } from "weysabi/ai-sdk";
import { RagEngine, RagManager } from "weysabi/rag";
import { ConversationMemory } from "weysabi/chat";
import { PromptRegistry } from "weysabi/prompts";
import { registerProvider } from "weysabi";
import type { ProviderHandler } from "weysabi";
import { WebSocketClient, createWebSocketClient } from "weysabi/client";

License

MIT