my-ai-api

A unified, provider-agnostic AI client for chat, streaming, image generation/editing, hosted web search, tool-calling, embeddings, model listing, realtime voice, and subscription usage/quota across OpenAI, Anthropic (Claude), Google (Gemini and Antigravity), and xAI (Grok) — with pluggable credential sources, including reuse of your Codex CLI, Claude Code, and Antigravity (agy) logins.

One neutral model in, one normalized result out. Swap providers by changing a string.

Capabilities by provider

| Capability | OpenAI | Anthropic | Gemini API | Antigravity (agy) | xAI | |---|:---:|:---:|:---:|:---:|:---:| | Chat generate / stream | ✅ | ✅ (native) | ✅ (compat) | ✅ (native) | ✅ (compat) | | Tool-call loop (run) | ✅ | ✅ | ✅ | ✅ | ✅ | | Images (generateImage / editImage) | ✅ + Codex login | — | ✅ Gemini + Imagen ¹ | ✅ nano-banana ² | ✅ Imagine | | Hosted web search (AI.search) | ✅ + Codex web.run | ✅ | ✅ | ✅ googleSearch ³ | ✅ + Grok Build login | | Models list | ✅ | ✅ | ✅ | ✅ | ✅ | | Embeddings | ✅ | — ⁴ | ✅ | — | ✅ | | Realtime voice | ✅ | — | — | — | — | | Ephemeral client secrets | ✅ | — | — | — | — | | Streaming transcription | ✅ | — | — | — | — | | Subscription usage/quota | ✅ ⁵ | ✅ ⁶ | — ⁷ | ✅ ⁸ | ◑ ⁹ |

¹ Gemini image models support generation and reference-image editing. The older Imagen :predict route is generation-only and deprecated by Google, but remains available until its announced shutdown. ² Antigravity has no dedicated image route: the nano-banana family (gemini-3.1-flash-image) generates and edits through generateContent with responseModalities: ["TEXT", "IMAGE"], and returns inline base64 only — never hosted URLs. ³ Antigravity grounds answers with the hosted googleSearch tool, which takes no arguments; domain and location filters have no equivalent and are rejected rather than silently dropped. ⁴ Anthropic has no first-party embeddings endpoint (they recommend Voyage AI). ⁵ ChatGPT/Codex login only — GET /backend-api/codex/usage on the Codex backend. ⁶ Claude Pro/Max OAuth login only — GET /api/oauth/usage (undocumented, RE-derived); reuses your existing Claude Code login, or ai-api login --provider claude. ⁷ A GEMINI_API_KEY is billing-metered and exposes no quota RPC. ⁸ agy OAuth login only — Code Assist retrieveUserQuotaSummary (undocumented, RE-derived). ⁹ xAI has no remaining-credit API; rate-limit budget from x-ratelimit-* headers, read via the inference-free POST /v1/tokenize-text. Works for an API key and a ~/.grok login.

import { AI } from "my-ai-api";

const ai = new AI();

// Codex CLI token (~/.codex/auth.json) is used automatically for OpenAI;
// other providers fall back to their API-key env vars.
const openai = ai.chat("openai");
const reply = await openai.generate({
  model: "gpt-5-mini",
  messages: [{ role: "user", content: "Say hi in five words." }],
});
console.log(reply.message.text);

Why

The realtime-voice code in gpt-2-realtime fused three concerns — auth, transport (WebSocket/HTTP), and provider (endpoints + wire format). ai-api separates them so any credential source can drive any provider over any transport, and new vendors are a single adapter folder.

Install

pnpm add my-ai-api

ESM-only, Node 22+.

Architecture

auth/         Credential sources → neutral AuthCredentials (Codex CLI, Claude Code, env keys, …)
core/         Neutral types, streaming event unions, typed error hierarchy
http/         fetch wrapper (timeout, retry, backoff) + SSE parser
transport/    Provider-agnostic authenticated WebSocket transport
providers/    Vendor adapters implementing Chat/Images/Search/Models/Embeddings/Usage/Voice capabilities
  provider.ts   the capability contracts every adapter implements
  headers.ts    shared `bearerHeaders` credential attachment
  openai/       chat/images (compat) + GPT-Realtime voice + client secrets +
                transcription + GPT-Live WebRTC + Codex-header usage
  anthropic/    native Messages API + models + OAuth usage
  google/       Gemini chat + native image generation/editing + Code Assist usage
  antigravity/  native Antigravity chat + nano-banana images + googleSearch
                grounding + Code Assist quota, on agy's existing OAuth login
  xai/          Grok chat + Imagine image generation/editing + key-status usage
  openai-compat/  shared /chat/completions, /models, /embeddings, /images adapters
chat/         ChatClient — unified generate/stream + automatic tool loop
voice/        VoiceClient — unified realtime voice over any VoiceCapability
tools/        ToolRegistry + opt-in bash / web-search tools
search/       Hosted-search request/result types + complete Codex web.run commands
images/       Neutral image request/reference/result/stream types
samples/      Runnable GPT-Realtime + GPT-Live examples
swift/        Swift Package (MyAIAPI) — same seams for iOS/macOS apps

Adding a provider

Every vendor folder follows the same shape, so a new provider is mechanical:

providers/<vendor>/
  config.ts   base URLs, constants, and the header builder — imported by every
              other file in the folder, so nothing has to import index.ts back
  chat.ts     ChatCapability      (or reuse openai-compat/chat.ts)
  models.ts   ModelsCapability    (or reuse openai-compat/models.ts)
  search.ts   SearchCapability
  usage.ts    UsageCapability
  realtime.ts VoiceCapability factory
  index.ts    re-exports the folder + `create<Vendor>Provider()` wiring

Two conventions the adapters rely on:

• Capability factories take one optional config object — create<Vendor><Capability>(config: <Vendor><Capability>Config = {}). Never positional arguments, so adding a knob is not a breaking change.
• config.ts is the leaf. Constants and header builders live there and are imported downward only. Putting them in index.ts creates a cycle as soon as a capability file needs them.

Provider is a bag of optional capabilities rather than a base class to extend: providers genuinely support different subsets (Anthropic has no embeddings, only OpenAI and xAI have voice), and callers get a real UnsupportedError plus type narrowing off the absent property.

Provider<TVoice> is generic in its realtime-voice option type, so createVoice is checked against the vendor's own options instead of taking Record<string, unknown>.

Vendor-specific surfaces

AI's methods are provider-neutral. Features with no cross-provider equivalent live behind a namespace accessor so the neutral surface does not grow one method per vendor:

const ai = new AI({ auth: codexAuth(), tools });

ai.chat("anthropic");          // neutral — any provider
ai.openai().gptLive();         // OpenAI-only — GPT-Live WebRTC
ai.openai().realtimeSession(); // OpenAI-only — ephemeral client secrets
ai.openai().transcriptionSession();

When a second vendor ships an equivalent, the helper graduates to a capability on Provider and gets a neutral AI method.

Swift package

Swift lives in swift/ and mirrors the same seams (Auth, Core, HTTP, Transport, Providers/OpenAI, Voice, Tools, AI) for the capabilities it covers. Tagged releases ship a root Package.swift so SPM can resolve the repo URL directly:

.package(url: "https://github.com/guocity/my-ai-api.git", from: "0.2.0")

It is deliberately narrower than the Node client, and covers OpenAI only — there is no Swift Anthropic, Google, or xAI adapter:

| Capability | Node | Swift | |---|:---:|:---:| | Realtime voice (GPT-Realtime) | ✅ | ✅ | | GPT-Live WebRTC call + delegation | ✅ | ✅ | | Tools (ToolRegistry, bash, web search) | ✅ | ✅ | | Provider-hosted web search | ✅ | — | | Codex CLI / env auth | ✅ | ✅ | | HTTP retries, backoff, typed errors | ✅ | ✅ | | Chat generate / stream + tool loop | ✅ | — | | Image generation / editing | ✅ | — | | SSE streaming | ✅ | — | | Models / embeddings / usage | ✅ | — | | Anthropic, Google, xAI providers | ✅ | — |

Behaviour is aligned where both platforms implement a capability — same realtime defaults, same read-only shell policy, same delegation replies, same call-id parsing. Anything in the "—" rows is Node-only for now.

The main seams that make it extensible:

• ChatCapability — generate() / stream() mapping the neutral model to a vendor. Adding a chat provider = one adapter; if the vendor is OpenAI-compatible, reuse createOpenAICompatChat.
• ImagesCapability — generate() / edit() plus optional streaming. OpenAI and xAI share the JSON Images wire adapter; Gemini maps the same neutral request to native generateContent media parts.
• VoiceCapability — connection(), initialMessages(), decode(), encode(). VoiceClient and the WebSocket transport never learn a vendor's event names.

Auth sources

Credentials normalize to AuthCredentials { token, accountId?, … }. Providers decide how to attach them (Bearer vs x-api-key, chatgpt-account-id, etc.).

import {
  agyAuth, antigravityAuth, antigravityDesktopAuth,
  codexAuth, grokAuth, claudeCodeAuth, claudeAuth,
  fromEnv, firstAvailable, apiKey,
} from "my-ai-api/auth";

codexAuth();                              // read ~/.codex/auth.json per request
agyAuth();                                // read agy's OAuth file per request
antigravityDesktopAuth();                 // desktop Keychain/file only
antigravityAuth({ source: "desktop-or-agy" }); // desktop first, then agy
grokAuth();                               // read ~/.grok/auth.json per request
claudeCodeAuth();                         // reuse the existing Claude Code login
claudeAuth();                             // this library's own Claude OAuth login
fromEnv("OPENAI_API_KEY");                // lazy env lookup
apiKey("sk-…");                           // explicit
firstAvailable(codexAuth(), fromEnv("OPENAI_API_KEY")); // try in order

Codex loading is read-only: it never performs an OAuth refresh. An expired token throws AuthExpiredError ("Auth token expired, need refresh…") so you can prompt codex login. Antigravity loading is read-only in both senses: agyAuth() re-reads ~/.gemini/antigravity-cli/antigravity-oauth-token per request, never copies or modifies it, and — since 0.1.9 — does not refresh by default either. Redeeming agy's refresh token can rotate the grant server-side and invalidate the copy agy still holds, so "in memory only" does not make it safe. An expired token throws AuthExpiredError naming agy models instead. The refresh option and implementation are retained for compatibility, but a hard safety gate prevents { refresh: true } from calling the OAuth token endpoint. Use { path } (or AGY_AUTH_PATH / ANTIGRAVITY_AUTH_PATH) to read a different file, and { projectId } to skip companion-project discovery.

Antigravity can also select { source: "desktop" | "agy" | "desktop-or-agy" } (or ANTIGRAVITY_AUTH_SOURCE). Desktop mode performs one narrow read of the exact Antigravity 2.4.x macOS Keychain item, then checks the desktop standalone store at ~/.gemini/jetski-standalone-oauth-token (override with ANTIGRAVITY_DESKTOP_AUTH_PATH). The running desktop language server reports whether it is signed in but intentionally does not expose its raw OAuth token; if its login exists only in memory, desktop therefore reports AuthMissingError and desktop-or-agy proceeds to agy's file. Fallback happens only for missing or expired desktop credentials—not malformed or unreadable ones—and neither source is ever refreshed by this library. Grok loading is also read-only: an expired or absent file throws AuthExpiredError / AuthMissingError telling you to run grok login. Check expiresAtMs rather than assuming a lifetime — a grok login OIDC token observed on 2026-07-30 was good for six hours, while the CLI's own docs quote seven days for a sign-in session token.

A ~/.grok login is accepted directly by api.x.ai — verified live on 2026-07-30 against /chat/completions, /models, /embeddings and /responses — so chat, models, embeddings and Imagine all use it, exactly as XAI_API_KEY does. Two routes are the exception. Hosted search deliberately uses Grok Build's CLI proxy (cli-chat-proxy.grok.com/v1) with X-XAI-Token-Auth: xai-grok-cli, because that is the contract Grok Build's own search runs on; note the proxy serves build-tuned variants and a narrower catalogue, so a grok-4.5 request there returns grok-4.5-build, and its /models lists one model where api.x.ai lists ten. And usage() reads account status from GET /v1/me, because GET /v1/api-key is the one xAI route that rejects this token. Gemini API-key auth remains available through GEMINI_API_KEY; Gemini CLI OAuth is not used.

Anthropic: two OAuth paths

Anthropic's default is firstAvailable(claudeCodeAuth(), claudeAuth(), fromEnv("ANTHROPIC_API_KEY")).

claudeCodeAuth() — reuse the Claude Code / Claude.app login (preferred). Resolves, in order, CLAUDE_CODE_OAUTH_TOKEN; the OS keychain item Claude Code-credentials under your username (read via security find-generic-password, as Claude Code itself does — its bundled keytar is a stub); then $CLAUDE_CONFIG_DIR/.credentials.json, default ~/.claude. A custom config dir shifts the keychain item to Claude Code-credentials-<8 hex>, the SHA-256 prefix of the NFC-normalized directory path.

This loader never writes and never refreshes, and that is deliberate: Anthropic rotates the refresh token on every refresh, so refreshing here would invalidate the one Claude Code still holds and break your login in the app — and we cannot write the replacement back into a keychain item we don't own. An expired token throws AuthExpiredError telling you to open Claude Code; there is no credential cache, so a background refresh it performs is picked up on the next request. Pass { requireFresh: false } to inspect a stale login anyway.

claudeAuth() — a login this library owns. ai-api login --provider claude runs OAuth2 + PKCE against claude.ai/oauth/authorize with Claude Code's public client id, serving the callback on the fixed http://localhost:54545/callback, and stores the result at ~/.ai-api/claude-oauth-token.json (mode 0600, override with CLAUDE_AUTH_PATH). Because this file is ours, claudeAuth() does refresh and write back. The on-disk shape matches CLIProxyAPI's ClaudeTokenStorage, so either tool can read the other's file. Headless boxes can paste the code#state the consent screen shows into completeClaudeLogin().

Caveat: CLIProxyAPI uses a Firefox uTLS fingerprint to get past Cloudflare on Anthropic domains, which Node's fetch cannot reproduce. If the token exchange fails, the error says so explicitly rather than blaming invalid JSON. claudeCodeAuth() is unaffected — it performs no token-endpoint call at all.

const agy = new AI().chat("antigravity"); // default auth is agyAuth()
const answer = await agy.generate({
  model: "gemini-3.1-pro-high",
  messages: [{ role: "user", content: "Explain this repository." }],
});

Note: a Codex/ChatGPT token authenticates the ChatGPT backend; whether it is accepted on every api.openai.com route depends on your account. For non-OpenAI providers use that vendor's API key.

Image generation and editing

AI.generateImage() and AI.editImage() normalize hosted URLs, inline base64, MIME types, revised prompts, and token usage. Image references can be public URLs, base64 data: URLs, or provider file IDs where supported.

import { AI } from "my-ai-api";

const ai = new AI();

const generated = await ai.generateImage("xai", {
  model: "grok-imagine-image-quality",
  prompt: "A cinematic mountain observatory above the clouds",
  aspectRatio: "16:9",
  size: "2K",
  responseFormat: "base64",
});
const base64 = generated.images[0]?.base64;
if (!base64) throw new Error("Imagine returned no inline image");

const edited = await ai.editImage("openai", {
  model: "gpt-image-2",
  prompt: "Keep the composition and change the scene to winter",
  images: [{ url: `data:image/jpeg;base64,${base64}` }],
  size: "auto",
  quality: "auto",
});

Gemini image models use the same methods with gemini-3.1-flash-image (or another image-capable Gemini model). Reference images become native inlineData or fileData parts. Legacy imagen-* models route to :predict and support generation only.

The same two methods work on antigravity, billed against your agy subscription instead of an API key:

const logo = await ai.generateImage("antigravity", {
  model: "gemini-3.1-flash-image",   // nano-banana
  prompt: "A minimalist flat-design logo of a purple gravity well",
  aspectRatio: "16:9",
  size: "2K",
});

const warmer = await ai.editImage("antigravity", {
  model: "gemini-3.1-flash-image",
  prompt: "Recolour the gravity well warm orange, keep everything else identical",
  images: [{ url: `data:${logo.images[0]!.mimeType};base64,${logo.images[0]!.base64}` }],
});

aspectRatio, size, outputFormat, and outputCompression map onto imageConfig; count becomes candidateCount. Antigravity returns inline base64 only, so responseFormat: "url", quality, background, and edit masks throw UnsupportedError.

OpenAI's Image API can also stream partial images:

for await (const event of ai.streamImage("openai", {
  model: "gpt-image-2",
  prompt: "A river made of white owl feathers",
  partialImages: 2,
})) {
  if (event.type === "partial") console.log(event.index, event.image.base64);
  else console.log(event.result.images[0]);
}

Vendor-only fields remain available through providerOptions. Unsupported common controls fail clearly rather than being silently ignored.

Hosted web search

AI.search() invokes the provider's own search/grounding tool and normalizes the answer, issued queries, URL citations, image results, usage, and raw response. The older createWebSearchTool() remains available as a local DuckDuckGo-backed function tool; it is a different, provider-independent fallback.

import { AI } from "my-ai-api";

const ai = new AI();

// Uses ~/.grok/auth.json and Grok Build's authenticated CLI proxy when logged
// in, with XAI_API_KEY/GROK_API_KEY as the fallback.
const result = await ai.search("xai", {
  model: "grok-4.5",
  query: "What changed in Node.js this week?",
  settings: {
    filters: { allowedDomains: ["nodejs.org", "github.com"] },
    enableImageSearch: false,
  },
});

console.log(result.text);
for (const source of result.sources) console.log(source.title, source.url);

The same call works with openai, anthropic, or google; use a model that supports that provider's hosted search. Common settings are mapped only where a provider supports them, while provider-only controls remain typed:

| Setting | OpenAI/Codex | Claude | Gemini API | Antigravity | xAI/Grok Build | |---|:---:|:---:|:---:|:---:|:---:| | allowed / blocked domains | ✅ | ✅ | via providerOptions | ✖ rejected | ✅ (max 5; mapped to excluded) | | approximate user location | ✅ | ✅ | via providerOptions | ✖ rejected | — | | context size / web-access mode | ✅ | — | — | — | — | | image result settings | ✅ | — | via grounding metadata | via grounding metadata | image search / understanding flags | | max uses / response inclusion / allowed callers | — | ✅ | — | — | — |

Antigravity's googleSearch tool takes no arguments, so filters and userLocation throw UnsupportedError instead of being dropped without a trace. Its answers carry queries, sources (with the cited span per source), and usage.

Current Codex also exposes a richer command endpoint. Supplying commands selects it and preserves the full web.run signature: searchQuery, imageQuery, open, click, find, PDF screenshot, finance, weather, sports, time, and responseLength.

const page = await ai.search("openai", {
  id: "research-session",
  model: "gpt-5",
  commands: {
    searchQuery: [{ q: "OpenAI platform updates", recency: 7, domains: ["openai.com"] }],
    open: [{ refId: "turn0search0", lineNumber: 20 }],
    responseLength: "medium",
  },
  settings: { externalWebAccess: "live", searchContextSize: "high" },
});

With a Codex login credential, command search routes through the ChatGPT Codex backend. Other providers reject commands clearly instead of silently dropping unsupported operations. providerOptions is merged last as a forward- compatibility escape hatch for newly released vendor fields.

Chat

const claude = new AI().chat("anthropic");            // Claude Code login, else ANTHROPIC_API_KEY
for await (const ev of claude.stream({ model: "claude-sonnet-5", messages })) {
  if (ev.type === "text.delta") process.stdout.write(ev.delta);
}

Tool calling

import { AI, ToolRegistry, createBashTool } from "my-ai-api";

const tools = new ToolRegistry([createBashTool({ cwd: "./workdir", mode: "readonly" })]);
const ai = new AI({ tools });
const { result, rounds } = await ai.chat("openai").run({
  model: "gpt-5",
  messages: [{ role: "user", content: "List files, then summarize." }],
});
// run() executes tool calls and loops until the model gives a final answer.

ToolCallPart.signature holds an opaque provider token that must be replayed verbatim with the call. Antigravity rejects a functionCall sent back without its thought_signature, so pass assistant messages through unchanged — the loop in run() already does. Never synthesize or copy one between calls.

Realtime voice

See also runnable examples under samples/:

pnpm sample:realtime
pnpm sample:realtime-tools
pnpm sample:gpt-live   # then open http://127.0.0.1:8787/

import { AI } from "my-ai-api";

// Audio output and the "marin" voice are the defaults; pass audio: false for a
// text-only session. transcribeInput is opt-in because Whisper bills per turn.
const voice = new AI().voice("openai", { transcribeInput: true });
voice.on("audio.delta", (pcm) => speaker.write(pcm));
voice.on("transcript.delta", (text, role) => console.log(role, text));
await voice.connect();
voice.appendAudio(micPcm16); // 24 kHz mono PCM

| Option | Default | Notes | |---|---|---| | audio | true | false gives a text-only session (no session.audio) | | voice | "marin" | Applies only when audio is enabled | | turnDetection | true | Server VAD | | transcribeInput | false | Opt in for user transcripts; billed per turn |

Repeated tool_call events for the same call id execute once — the client tracks handled ids for the life of the connection.

Browser clients: ephemeral secrets

A browser must never hold a Codex token or API key. Mint a session-scoped ek_... secret server-side and hand only that to the client. Tool definitions are tagged with the Realtime wire type on the way out, so the same ToolRegistry drives both server sockets and browser sessions.

import { AI, codexAuth } from "my-ai-api";

const sessions = new AI({ auth: codexAuth(), tools }).openai().realtimeSession({
  defaultInstructions: "You are the voice assistant for Acme.",
  sessionTools: [docs],                        // optional hosted MCP/connectors
  safetyIdentifier: hashInternalUserId(user.id),
});

// Return this to the browser; it expires on its own.
const { value, expiresAt, model } = await sessions.mint({ voice: "marin" });

buildSession() returns the payload without minting, for inspection or a custom transport. A failed mint throws ClientSecretError carrying status and the response body.

Streaming transcription

A transcription session produces no assistant turn — push PCM in, read item-aware transcript events out. gpt-realtime-whisper has no server VAD, so utterances are closed with an explicit commitAudio().

import { RealtimeTranscriptionClient, codexAuth } from "my-ai-api";

const stt = new RealtimeTranscriptionClient({
  auth: codexAuth(),
  language: "en",
  delay: "low",                                 // minimal | low | medium | high | xhigh
  include: ["item.input_audio_transcription.logprobs"],
});
stt.on("transcript.delta", ({ delta, itemId }) => console.log(itemId, delta));
stt.on("transcript.completed", ({ transcript }) => console.log(transcript));

await stt.connect();
stt.appendAudio(micPcm16);
stt.commitAudio();

Whisper's constraints (24 kHz mono PCM, no turn detection, no prompts) are validated locally by buildTranscriptionSession(), so a misconfiguration throws TranscriptionConfigError up front instead of failing as an opaque server rejection mid-stream. Non-whisper models skip those checks. For browser clients, ai.openai().transcriptionSession() mints ephemeral transcription secrets the same way.

Experimental GPT-Live WebRTC

GptLiveClient is the app-facing entry point: Codex SDP call creation plus the oai-events client-delegation protocol. This library does not create RTCPeerConnection itself — browsers, Electron, or a native WebRTC stack own media. Your app plugs Live into that peer.

import {
  AI,
  ToolRegistry,
  createBashTool,
  createWebSearchTool,
  codexAuth,
} from "my-ai-api";

const tools = new ToolRegistry([
  createBashTool({ cwd: process.cwd(), name: "run_shell", mode: "readonly" }),
  createWebSearchTool(),
]);

const live = new AI({ auth: codexAuth(), tools }).openai().gptLive();

// 1) App creates RTCPeerConnection, adds mic tracks, opens data channel "oai-events"
const offer = await peerConnection.createOffer();
await peerConnection.setLocalDescription(offer);

// 2) Exchange SDP through this library (trusted backend)
const call = await live.createCall({
  sdp: offer.sdp!,
  voice: "cove",
  instructions: "Talk naturally and keep answers concise.",
});
await peerConnection.setRemoteDescription({ type: "answer", sdp: call.sdp });

// 3) Forward data-channel messages; send replies back on the same channel
eventsChannel.onmessage = async (message) => {
  const { replies } = await live.handleDataChannelMessage(String(message.data));
  for (const reply of replies) eventsChannel.send(JSON.stringify(reply));
};

Lower-level pieces (same exports):

• GptLiveCallService — SDP offer → answer (retries without session.model when Codex rejects it)
• GptLiveDelegationService — natural-language task → Realtime tool agent → answer
• Protocol helpers — parseDelegationTask, buildDelegationReplies, GPT_LIVE_EVENTS_CHANNEL

This targets gpt-live-1-boulder-alpha through an undocumented ChatGPT/Codex backend route. It requires an eligible Codex/ChatGPT login and may change without notice; keep the access token on a trusted backend, not in browser code.

Protocol versions

The realtime wire has three incompatible versions, and GPT-Live is v3 ("Frameless Bidi"). Everything that differs between them is derived from one RealtimeVersion rather than hardcoded, so the same client can speak all three:

| | v1 | v2 | v3 (GPT-Live) | |---|---|---|---| | Default model | gpt-realtime-1.5 | gpt-realtime-1.5 | gpt-live-1-boulder-alpha | | openai-alpha header | quicksilver=v1 | (none) | quicksilver=v2 | | Session shape | type: "quicksilver" + input audio format | type: "realtime" + output_modalities | delegation: {type:"client"} + optional initial_items | | Voices | 9 quicksilver voices, default cove | 10 realtime voices, default marin | same 9 as v1 | | Text output modality | ✗ | ✅ | ✗ | | WebRTC (AVAS) calls | ✅ | ✗ | ✅ |

Note that the alpha header does not track the version number: v3 sends quicksilver=v2, because the header names the quicksilver architecture revision. A WebRTC call rejects v2 outright.

import {
  buildRealtimeCallUrl,
  buildRealtimeSessionJson,
  contextAppendChunks,
  parseRealtimeEvent,
} from "my-ai-api";

// Route and query are derived, not hardcoded: the ChatGPT backend route posts to
// realtime/calls with the AVAS query, while the plain API route serves frameless
// sessions from `live` with no query and a multipart body.
buildRealtimeCallUrl("https://chatgpt.com/backend-api/codex", "v3");
// → …/realtime/calls?intent=quicksilver&architecture=avas
buildRealtimeCallUrl("https://api.openai.com/v1", "v3"); // → …/v1/live

// Inbound events normalize to one union regardless of version.
parseRealtimeEvent({ type: "turn.done", turn: { role: "user", transcript: "hi" } }, "v3");
// → { type: "inputTranscriptDone", text: "hi" }

// Context appends are chunked at 500 bytes, never truncated, so a long tool
// trace reaches the model whole.
contextAppendChunks(longToolOutput); // → string[]

buildRealtimeSidebandUrl(baseUrl, callId, version) builds the control-socket URL for an existing WebRTC call — /v1/live/<callId> for v3, or a call_id query parameter for v1. A client that opened an oai-events data channel already has that path and does not need it.

Omitting the append channel is meaningful: it leaves the decision of whether to speak to the model, which is the default upstream behaviour. Passing speakable forces narration, and commentary makes the text silent context.

Models & embeddings

const ai = new AI();

const models = await ai.models("openai");            // ModelInfo[] (GET /models)
const { embeddings } = await ai.embed("openai", {
  model: "text-embedding-3-small",
  input: ["hello", "world"],                          // string | string[]
});
embeddings[0].values;                                 // number[] (base64 auto-decoded)

models() and embed() throw UnsupportedError on providers that lack the surface (e.g. embed("anthropic")).

Antigravity model listing merges the app's separate imageGenerationModelIds list into the model dictionary, so gemini-3.1-flash-image is returned alongside the selectable chat models.

Subscription usage / quota

"How much is left" for a subscription login, normalized across providers into UsageSnapshot { provider, plan?, windows[] }. Each UsageWindow carries any of usedPercent, used/limit/remaining, unit, and resetsAtMs.

const snap = await ai.usage("openai");   // needs a ChatGPT/Codex login, not an API key
for (const w of snap.windows) {
  console.log(w.label, w.usedPercent, w.resetsAtMs && new Date(w.resetsAtMs));
}

Sources per provider (all subscription-only — plain API keys throw UnsupportedError pointing you at the billing dashboard):

• OpenAI — GET https://chatgpt.com/backend-api/codex/usage (ChatGPT/Codex OAuth login; a plain API key is rejected before the request). One cheap GET that spends no plan quota. parseCodexUsageResponse(body) is exported.

  Verified against a live response on 2026-07-30. A Plus account reports a single seven-day primary_window and sends secondary_window: null, so the window count is plan-dependent; credits.balance is a decimal string ("0"), and an unlimited account reports no meaningful number, so that case is labelled without a remaining.

  The same budget also rides on POST /backend-api/codex/responses as x-codex-* response headers and as a mid-stream rate_limits SSE event. parseCodexRateLimitHeaders(headers) and parseCodexRateLimitEvent(payload) are exported for those, so a caller already streaming a completion can read the budget off that response instead of issuing a second request.

• Anthropic — GET https://api.anthropic.com/api/oauth/usage (Claude Pro/Max OAuth token; an sk-ant-api… key is rejected before the request). This endpoint is undocumented and reverse-engineered; field names may drift, so parsing is tolerant and snapshot.raw holds the original body. parseAnthropicUsage(body) is exported.

  Verified against a live response on 2026-07-30. The body carries the same windows twice — as top-level five_hour / seven_day objects keyed on utilization, and as a limits[] array keyed on percent with a kind label — plus a wide set of null-valued windows for plans the account does not have. The parser reads both spellings, labels limits[] entries by kind, and drops an entry that restates a window already emitted (matching on percentage and a reset within the same minute, since the two spellings stamp their timestamps microseconds apart). extra_usage is reported only when actually enabled: a disabled account still sends monthly_limit: null. The response carries no plan field, so snapshot.plan falls back to the subscriptionType recorded in a Claude Code credential.

• Antigravity — agy OAuth login via Code Assist: loadCodeAssist (tier) → retrieveUserQuotaSummary (grouped five-hour/weekly buckets). The companion project comes from the credential when agyAuth() supplied one, so the tier call is not trusted to name it. parseGeminiQuota(buckets) is retained for the older per-model response; parseAntigravityQuotaSummary() parses the current grouped response. Not offered on the google provider: the Gemini API is billing-metered, and these RPCs only accept a subscription OAuth token.

• xAI — rate-limit budget plus account status; works for an XAI_API_KEY and a ~/.grok/auth.json login. Verified live 2026-07-30.

  There is no subscription-quota endpoint (credit balance lives in the console), and every candidate route 404s: /v1/usage, /v1/credits, /v1/billing, /v1/limits, /v1/quota, the same names on the CLI proxy, and management-api.x.ai. What xAI does expose is the live rate-limit budget in x-ratelimit-* headers on inference responses. usage() reads those from POST /v1/tokenize-text, the one route that returns them without running inference — so asking for your budget does not spend it. That yields a requests window (limit, remaining, used, usedPercent); the tokens window only appears on real completions, so pass their headers to the exported parseXaiRateLimitHeaders(headers) to get both. Note the reset headers are durations ("2m30s"), not epochs, and are converted accordingly.

  Account status comes from whichever identity route accepts the credential: GET /v1/api-key for a key (plan: "api" / "blocked"), GET /v1/me for a ~/.grok login — that login is rejected by /api-key specifically (401 "API key is missing.") though api.x.ai accepts it everywhere else. For a login, plan is the tier claim on the token (e.g. "tier-3"), which is the rate-limit tier the windows above are graded against. GET https://grok.com/rest/subscriptions also reports a consumer tier (SUBSCRIPTION_TIER_X_PREMIUM) but describes an X subscription rather than the API budget, so it is not used.

These internal endpoints are not officially supported by the vendors and can change without notice. Treat usage() as best-effort telemetry.

CLI

ai-api providers                          # list providers
ai-api auth                               # describe Codex auth (masked)
ai-api auth --provider antigravity        # validate agy auth (masked; no disk writes)
ai-api auth --provider antigravity --auth-source desktop-or-agy
ai-api auth --provider antigravity --auth-source desktop-or-agy --refresh-auth # retained, refresh disabled
ai-api auth --provider antigravity --path ./token   # read a specific credential file
ai-api auth --provider claude-code        # describe the existing Claude Code login
ai-api login --provider claude            # OAuth + PKCE sign-in this library owns
ai-api usage --provider anthropic         # Claude Pro/Max window utilization
ai-api usage --provider anthropic --json  # the untouched provider body
ai-api models --provider antigravity      # list models available to the agy login
ai-api models --provider antigravity --auth-source desktop-or-agy
ai-api usage --provider antigravity       # remaining Code Assist quota per model
ai-api chat --provider antigravity --model gemini-3.1-pro-high "hello"
ai-api chat --provider anthropic --model claude-sonnet-5 "hello"
ai-api chat --provider openai --model gpt-5 --stream "stream this"
ai-api models --provider openai           # list available models
ai-api embed --provider openai --model text-embedding-3-small "embed me"
ai-api usage --provider openai            # remaining subscription quota

Adding a provider

1. Create src/providers/<vendor>/index.ts exporting create<Vendor>Provider().
2. Implement chat (reuse createOpenAICompatChat if compatible) and/or createVoice.
3. Register it in defaultProviders().

No changes to ChatClient, VoiceClient, transports, or auth.

Develop

pnpm install
pnpm build        # tsc → dist/
pnpm test         # node:test
pnpm typecheck

License

MIT