@kenkaiiii/gg-ai

One function. Flat options. Switch providers by changing a string. No adapters, no plugins, no wrapper classes.

Part of the GG Framework monorepo.

Install

npm i @kenkaiiii/gg-ai

How it works

Call stream() with a provider, model, and messages. That's the entire API.

• for await gives you streaming events (text_delta, thinking_delta, toolcall_done, etc.)
• await gives you the final response (message, stopReason, usage)

Same function, same call. Dual-nature StreamResult — async iterable and thenable.

Tool parameters are Zod schemas. Converted to JSON Schema at the provider boundary automatically.

Providers

| Provider | Models | Notes | |---|---|---| | anthropic | Claude Fable 5.1, Opus 5.5, Sonnet 5.5, Haiku 5.5 | Extended thinking, prompt caching, server-side compaction | | openai | GPT-6 Astra, GPT-6.1 Sol, GPT-6 Luna | OAuth (Codex endpoint) or API key | | gemini | Gemini 3.1 Pro (Preview), 3.8 / 3.7 / 3.5 Flash, 3.5 / 3.1 Flash Lite | OAuth (Code Assist); native video input | | xai | Grok 4.7 | OpenAI-compatible, https://api.x.ai/v1 | | moonshot | Kimi K3, K2.8 Preview, K2.7 Code, K2.7 Code HighSpeed | OpenAI-compatible; native video input | | glm | GLM-5.3, GLM-5.3-Flash | Z.AI coding endpoint, OpenAI-compatible | | minimax | MiniMax M3 | Anthropic-compatible endpoint | | xiaomi | MiMo-V2.6-Pro, MiMo-V2.6-Flash, MiMo-V2.6-Pro-UltraSpeed | OpenAI-compatible | | deepseek | DeepSeek V4 Pro, V4.1 Flash | OpenAI-compatible | | sakana | Fugu, Fugu Max, Fugu Ultra | OpenAI-compatible | | openrouter | Qwen3.8 Max, or any OpenRouter model id | OpenAI-compatible gateway | | huggingface | Kimi K2.7 Code, DeepSeek V4.1 Flash, GPT-OSS 120B | Inference Providers router, OpenAI-compatible | | local | Any model your server exposes | Ollama, LM Studio, llama.cpp, vLLM; baseUrl is required |

The model lists are the ones GG Coder ships in its registry (@kenkaiiii/gg-core). stream() passes model straight through, so any id the provider accepts works. Every provider's default endpoint can be overridden with baseUrl, and providerRegistry.register() adds your own.

Stream events

| Event | Description | |---|---| | text_delta | Incremental text output | | thinking_delta | Reasoning output, from any provider that streams it | | toolcall_delta | Streaming tool call arguments | | toolcall_done | Completed tool call with parsed args | | server_toolcall | Server-side tool invocation | | server_toolresult | Server-side tool result | | keepalive | Provider heartbeat: the stream is alive but has no new content yet | | done | Stream finished, includes stop reason | | error | Error occurred |

Options

| Option | Type | Description | |---|---|---| | provider | Provider (any id in the table above) | Required | | model | string | Required | | messages | Message[] | Required | | tools | Tool[] | Tool definitions with Zod schemas | | toolChoice | "auto" \| "none" \| "required" \| { name } | Tool selection strategy | | serverTools | ServerToolDefinition[] | Server-side tool definitions | | maxTokens | number | Max output tokens | | temperature | number | Sampling temperature | | topP | number | Nucleus sampling | | stop | string[] | Stop sequences | | thinking | "low" \| "medium" \| "high" \| "xhigh" \| "max" \| "ultra" | Reasoning effort; each provider maps it to what the model supports | | apiKey | string | Provider API key | | baseUrl | string | Custom endpoint | | signal | AbortSignal | Cancellation | | cacheRetention | "none" \| "short" \| "long" | Prompt cache preference | | promptCacheKey | string | Stable cache routing key (OpenAI, Moonshot, Gemini) | | webSearch | boolean | Provider-native web search where supported | | compaction | boolean | Server-side compaction (Anthropic only) | | clearToolUses | boolean | Server-side clearing of old tool results (Anthropic only) | | fetch | typeof fetch | Custom fetch, e.g. for React Native | | providerOptions | Record<string, unknown> | Provider-specific body fields; see below |

Provider-specific body fields

providerOptions is merged into the JSON body of stream(), embed() and rerank() requests. It only adds fields gg-ai hasn't set. Core fields (model, messages, input, query, documents, stream, tools, system, contents, requests, dimensions, encoding_format, top_n) can't be set this way, even when gg-ai left them out. The main use is OpenRouter's provider routing:

import { stream, type OpenRouterProviderPreferences } from "@kenkaiiii/gg-ai";

const result = stream({
  provider: "openrouter",
  model: "anthropic/claude-sonnet-4.5",
  messages,
  apiKey,
  providerOptions: {
    provider: { zdr: true, data_collection: "deny" } satisfies OpenRouterProviderPreferences,
  },
});

@kenkaiiii/gg-agent passes providerOptions and fetch through to every stream() call it makes.

Embeddings and reranking

embed() and rerank() use the same provider registry as stream().

import { embed, rerank } from "@kenkaiiii/gg-ai";

const { embeddings, usage } = await embed({
  provider: "openrouter",
  model: "google/gemini-embedding-2",
  input: chunks, // any length; split into batches for you
  dimensions: 1536,
  normalize: true,
  apiKey,
  providerOptions: { provider: { zdr: true, data_collection: "deny", allow_fallbacks: true } },
});
// embeddings[i] is the vector for chunks[i]

const { results } = await rerank({
  provider: "openrouter",
  model: "cohere/rerank-v3.5",
  query: "capital of France",
  documents,
  topN: 5,
  apiKey,
});
// results: [{ index, score }], best first; documents[results[0].index] is the top hit

What embed() guarantees

• Vectors come back in input order, even when the provider returns them out of order (gg-ai sorts by index).
• Long inputs are split at the provider's per-request limit (table below; override with batchSize). Batches run one at a time, in order.
• The response is checked: exactly one vector per input, all numbers, all the same length, and dimensions long when you asked for that. If any check fails it throws ProviderError. Nothing is padded or truncated.
• normalize: true L2-normalizes every vector. It's off by default. OpenAI embeddings come back unit-length already, and so does Gemini at full 3072 dimensions. gemini-embedding-2 also normalizes shortened vectors, but gemini-embedding-001 doesn't: at reduced dimensions its vectors are not unit-length, so pass normalize: true. Through OpenRouter or a local server, it depends on the upstream model. If you compare with cosine similarity or a dot product and aren't sure, normalize.
• inputType: "query" | "document" maps to the provider's native task parameter: Gemini taskType (RETRIEVAL_QUERY / RETRIEVAL_DOCUMENT) and OpenRouter input_type (search_query / search_document). It's ignored by OpenAI and local servers, which have no such parameter. It's also ignored for gemini-embedding-2, where Google wants the task written into the text instead (task: search result | query: …).
• An empty input returns { embeddings: [] } without making a request.

What rerank() guarantees: results are sorted by score, highest first, and each index points into your documents. At most topN results come back, and every index is checked. An empty documents returns { results: [] } without making a request.

Provider support

| Provider | embed | Max inputs per request | inputType | rerank | |---|---|---|---|---| | openai | ✓ /v1/embeddings | 2048 | ignored | — | | openrouter | ✓ /api/v1/embeddings | 64 (gg-ai default; OpenRouter documents no limit) | input_type | ✓ /api/v1/rerank | | gemini | ✓ native batchEmbedContents | 100 | taskType | — | | local | ✓ {baseUrl}/embeddings (Ollama, LM Studio, llama.cpp, vLLM) | 64 (gg-ai default) | ignored | ✓ {baseUrl}/rerank (llama.cpp --rerank, vLLM; Ollama and LM Studio have none and return 404) | | palsu | ✓ deterministic fake | all | ignored | ✓ deterministic fake | | anthropic, xiaomi, glm, moonshot, minimax, deepseek, sakana, xai, huggingface | — | | | — |

Calling embed() or rerank() on a provider without that capability throws GGAIError with source: "capability". Check first with providerRegistry.supports(name, "embed").

Notes:

• Gemini embeddings use the Gemini API with an AI Studio key, sent as x-goog-api-key. The Code Assist sign-in that stream() uses for Gemini chat has no embeddings endpoint.
• OpenAI embeddings need a platform API key. ChatGPT-subscription (Codex) tokens can't call /embeddings.
• Anthropic has no embeddings API (it points to Voyage AI). The other providers in the last row don't document an embeddings or rerank endpoint on the API gg-ai talks to, so none is registered for them.
• Document parsing and OCR are out of scope for now. That would be a separate API (file in, pages out), and providers disagree on it much more than on embeddings.

Errors, retries and cancellation

Failures use the same error model as stream():

| Failure | Error | |---|---| | HTTP 401 / 403 | ProviderError, source: "auth" | | HTTP 402, or billing text such as insufficient_quota | ProviderError with "usage limit reached" (isUsageLimitError() is true) | | HTTP 429 | ProviderError, statusCode: 429, resetsAt from Retry-After when sent | | Other HTTP errors | ProviderError with statusCode and requestId | | Connection failure | GGAIError, source: "network" | | signal aborted | AbortError (DOMException); remaining batches are skipped |

Provider error bodies are trimmed to their message, then redacted and truncated, so API keys never end up in an error message.

gg-ai doesn't retry. stream() doesn't either: retry policy lives in the caller, which for chat is @kenkaiiii/gg-agent. Retry 429s and 5xx with backoff if you need to. Never retry auth errors or anything where isUsageLimitError(err) is true.

Custom providers

A provider registered at runtime can offer embeddings and reranking too. stream is still required, and the other two are optional. gg-ai splits input into batches of embedBatchSize, validates and orders the results, normalizes, and sorts. Your functions only handle one request.

import { providerRegistry, embed, type Provider, type ProviderEntry } from "@kenkaiiii/gg-ai";

providerRegistry.register("my-search", {
  stream: () => { throw new Error("chat not supported"); },
  embedBatchSize: 128,
  embed: async ({ model, input, dimensions, apiKey, signal, fetch = globalThis.fetch }) => {
    const res = await fetch("https://example.com/embed", {
      method: "POST",
      headers: { Authorization: `Bearer ${apiKey}`, "Content-Type": "application/json" },
      body: JSON.stringify({ model, texts: input, dimensions }),
      signal,
    });
    const json = await res.json();
    return { embeddings: json.vectors, usage: { inputTokens: json.tokens } };
  },
  rerank: async ({ query, documents }) => ({
    results: documents.map((doc, index) => ({ index, score: doc.includes(query) ? 1 : 0 })),
  }),
} satisfies ProviderEntry);

// `Provider` lists the built-ins, so custom names need a cast (same as for stream()).
await embed({ provider: "my-search" as Provider, model: "v1", input: ["hello"] });

Testing without a network

registerPalsuProvider() gives palsu deterministic embed and rerank fakes. Embeddings are hashed bag-of-words unit vectors (64 dimensions by default, or dimensions), so texts that share words score closer together. Rerank scores are the fraction of query words found in each document. embedDimensions and embedBatchSize in the config change the defaults, and state.embedCallCount / state.rerankCallCount count requests.

License

MIT