@zhivex-ai/gateway

Routing and fallback package for Zhivex AI SDK.

The gateway now supports:

• generate()
• streamText()
• generateObject()
• streamObject()
• runAgent()
• streamAgent()

Tool loops continue to run on the selected target after routing, and streaming fallbacks are resolved before the first chunk is emitted.

For agent routing, the gateway can also filter by agentCapabilities, such as provider support tier or approval-capable MCP support, before selecting the final target.

Install

bun add @zhivex-ai/gateway

Usage

import { createGateway } from "@zhivex-ai/gateway";
import { createOpenAI } from "@zhivex-ai/openai";
import { createOllama } from "@zhivex-ai/ollama";

const gateway = createGateway({
  adapters: {
    openai: createOpenAI({ apiKey: process.env.OPENAI_API_KEY }),
    ollama: createOllama()
  },
  maxRetries: 1
});

const result = await gateway.generate({
  primary: { provider: "openai", modelId: "gpt-4o-mini" },
  fallbacks: [{ provider: "ollama", modelId: "llama3.2" }],
  messages: [{ role: "user", content: "Summarize the benefits of fallback routing." }],
  routingMode: "balanced"
});

console.log(result.text);
console.log(result.providerUsed);
console.log(result.attempts);

The gateway also supports streamText(), generateObject(), and streamObject() while preserving the selected target for the full request lifecycle, including tool loops. Object routes skip incompatible targets before making a provider call: native mode requires structuredOutput, prompted mode requires jsonMode, and auto mode accepts either capability.

For agent workloads, use runAgent() or streamAgent() to route by both regular model capabilities and agent-specific capabilities such as supportTier, approvalRequests, or remoteMcp.

const agentResult = await gateway.runAgent({
  primary: { provider: "kimi", modelId: "kimi-k2.5" },
  fallbacks: [{ provider: "qwen", modelId: "qwen-plus" }],
  prompt: "Use hosted retrieval and an MCP map server.",
  requiredAgentCapabilities: {
    supportTier: "tier-b",
    hostedFileSearch: true,
    remoteMcp: true
  }
});

console.log(agentResult.providerUsed);
console.log(agentResult.attempts);
console.log(agentResult.routeDecision);

This package is the SDK-local routing layer. It is not the Zhivex-hosted Gateway API and it is not re-exported from @zhivex-ai/sdk.

Repository and full documentation:

• https://github.com/Zhivex/zhivex-ai-sdk