@x12i/ai-router-runtime

Unified AI router runtime. Import only from this package — do not depend on @x12i/openrouter-runtime directly.

Call any configured provider through one protocol (the OpenRouter runtime request/response shape). Implemented providers today:

| Provider | run() | executeStreamingChat() | Function / server tools | | --- | --- | --- | --- | | openrouter | Yes | Yes | Yes | | llama-cpp | Yes (text) | PROVIDER_CAPABILITY_UNSUPPORTED | Unsupported | | transformersjs | Yes (text) | PROVIDER_CAPABILITY_UNSUPPORTED | Unsupported |

Other cloud provider ids (openai, anthropic, …) remain typed but throw PROVIDER_NOT_IMPLEMENTED.

Install

npm install @x12i/ai-router-runtime

Optional local engines

npm install node-llama-cpp              # provider: "llama-cpp"
npm install @huggingface/transformers  # provider: "transformersjs"

Missing optional deps throw a stable AiRouterError with code MISSING_OPTIONAL_DEP.

Usage

import { createAiRouterRuntime } from "@x12i/ai-router-runtime";

const router = createAiRouterRuntime({
  provider: "openrouter",
  openrouter: {
    apiKey: process.env.OPENROUTER_API_KEY!,
    defaults: {
      serverTools: {
        datetime: { mode: "allowed", timezone: "Asia/Jerusalem" }
      }
    }
  }
});

const response = await router.run({
  model: "openai/gpt-5.2",
  messages: [{ role: "user", content: "What time is it?" }]
});

console.log(response.text);

Local providers

const llama = createAiRouterRuntime({
  provider: "llama-cpp",
  llamaCpp: {
    modelPath: process.env.LLAMA_CPP_MODEL_PATH!, // or set env only
    contextSize: 4096,
    threads: 4
  }
});

await llama.run({ prompt: "Hello" });

const tf = createAiRouterRuntime({
  provider: "transformersjs",
  transformersjs: {
    modelId: process.env.TRANSFORMERS_JS_MODEL_ID!, // or set env only
    cacheDir: "/tmp/tf-cache"
  }
});

await tf.run({ prompt: "Hello" });

Env fallbacks: LLAMA_CPP_MODEL_PATH, LLAMA_CPP_CONTEXT_SIZE, LLAMA_CPP_THREADS, TRANSFORMERS_JS_MODEL_ID, TRANSFORMERS_JS_CACHE_DIR.

Per-request provider

const router = createAiRouterRuntime({
  provider: "openrouter",
  openrouter: { apiKey: process.env.OPENROUTER_API_KEY! },
  llamaCpp: { modelPath: "/models/model.gguf" }
});

await router.run({ provider: "openrouter", prompt: "Hello" });
await router.run({ provider: "llama-cpp", prompt: "Hello" });

// Throws PROVIDER_NOT_IMPLEMENTED
await router.run({ provider: "openai", prompt: "Hello" });

Public surface (import from this package)

Values

| Export | Notes | | --- | --- | | createAiRouterRuntime | Consumer entrypoint (do not use createOpenRouterRuntime) | | AiRouterError | Router-level errors | | OpenRouterHttpError | Re-exported for instanceof | | RuntimeConfigError | Re-exported for instanceof | | IMPLEMENTED_AI_PROVIDERS / isImplementedAiProvider / resolveProvider | Provider helpers |

Types (protocol)

RuntimeRequest, RuntimeResponse, RuntimeMessage, RuntimeFunctionToolDefinition, RuntimeDefaults, RuntimeLogger, RuntimeStreamEvent, CompiledOpenRouterRequest, OpenRouterRuntimeOptions

Types (server tools / results)

RuntimeServerToolsPolicy, WebSearchPolicy, WebFetchPolicy, DatetimePolicy, ImageGenerationPolicy, ApplyPatchPolicy, FusionPolicy, AdvisorPolicy, SubagentPolicy, ServerToolKey, ServerToolUsage, RuntimeCitation, RuntimeGeneratedImage, RuntimePatchProposal, RuntimeToolUsage

Router types

AiRouterRuntime, AiRouterRuntimeOptions, AiRouterRequest, AiProviderId, LlamaCppProviderOptions, TransformersJsProviderOptions

Entrypoints

| Method | Purpose | | --- | --- | | router.run(request) | Non-streaming call | | router.executeStreamingChat(request) | Streaming chat (OpenRouter only) | | router.compile(request) | Compile OpenRouter wire payload without sending |

run() / compile() always produce non-streaming bodies (stream: false). Streaming is only through executeStreamingChat().

Logging

Pass logger on the router (or on openrouter options). Lifecycle events:

• ai-router.request.started
• ai-router.request.succeeded (provider, entrypoint, requestId, status)
• ai-router.request.failed (provider, entrypoint, requestId, errorCode, message)
• ai-router.request.compiled (debug)

OpenRouter package console logging still respects OPENROUTER_RUNTIME_LOGS=true.