@odla-ai/ai

⚠️ Early access — pre-1.0. Agents work from bounded runbooks; humans approve credentials, production changes, releases, and merges. APIs and exact package availability can change. Review the documented guarantees and limitations; this software is MIT-licensed and provided without warranty.

One general-purpose interface for AI inference across Anthropic (Claude), OpenAI (GPT), and Google (Gemini) — text, image, and audio — plus a tool-use agent engine and an eval harness. It's the place every odla app reaches for when it needs AI.

• Library surface. Import it in-process and bring your own keys; this package exports no generic hosted gateway. Separately, odla's System AI broker uses the same facade server-side for bounded o11y/security purposes and returns purpose grants rather than provider credentials.
• One canonical shape. Anthropic-flavored content blocks, forced tool calls with object inputs, a normalized streaming event vocabulary, cache-aware usage, and a closed error taxonomy — the same regardless of which provider answers.
• Capability-checked. Every request is validated against the model before dispatch, so "audio to Claude" fails fast with a clear error instead of an opaque provider 400.
• Server runtimes. Node 20+ and Cloudflare Workers. Provider adapters are runtime-lazy, but all three provider SDKs are package dependencies. Do not put long-lived provider keys in browser code; use your own authenticated proxy.

npm install @odla-ai/ai

Quick start

import { init } from "@odla-ai/ai";

const ai = init({
  keys: {
    anthropic: process.env.ANTHROPIC_API_KEY,
    openai: process.env.OPENAI_API_KEY,
    google: process.env.GOOGLE_API_KEY,
  },
  defaultModel: "claude-opus-4-8",
});

// One call, any provider — pick the model, the provider is resolved for you.
const res = await ai.chat({
  model: "gpt-5",
  messages: [{ role: "user", content: "Say hello in one word." }],
  maxTokens: 64,
});
console.log(res.content); // [{ type: "text", text: "Hello" }]

Streaming

for await (const ev of ai.stream({ model: "gemini-2.5-flash", messages: [...], maxTokens: 512 })) {
  if (ev.type === "content_block_delta" && ev.delta.type === "text_delta") {
    process.stdout.write(ev.delta.text);
  }
}

The event union (message_start → content_block_start → content_block_delta → content_block_stop → message_delta → message_stop, plus error) is identical across providers.

Structured extraction (forced tool call)

const { value } = await ai.extract<{ sentiment: "positive" | "negative" | "neutral" }>({
  model: "claude-haiku-4-5",
  user: "Classify: 'I love this!'",
  tool: {
    name: "classify",
    parameters: {
      type: "object",
      properties: { sentiment: { type: "string", enum: ["positive", "negative", "neutral"] } },
      required: ["sentiment"],
    },
  },
});
// value.sentiment === "positive" (parsed and validated before it is returned)

extract and local agent tools validate provider arguments before returning or executing them. The dependency-free validator supports the common tool-schema subset: local $ref/$defs, type, enum, const, composition, object properties/required/additional properties, arrays, and string/number constraints. Unknown or unsupported schema keywords outside that documented subset fail closed. extract() throws ToolInputError; local agent tools return an error tool_result and their handlers do not execute.

Web search

const { text } = await ai.search({ model: "claude-sonnet-5", user: "Who won the 2026 World Cup?" });

Multimodal: text, image, audio

Content blocks carry the modality. Image is broadly supported; audio is accepted by OpenAI and Google, but not Anthropic — sending an audio block to a Claude model throws a CapabilityError before any network call.

// Image (works on Claude, GPT, Gemini)
await ai.chat({
  model: "gpt-4o",
  messages: [{ role: "user", content: [
    { type: "image", source: { type: "base64", mediaType: "image/png", data: pngB64 } },
    { type: "text", text: "What's in this image?" },
  ] }],
  maxTokens: 128,
});

// Audio (OpenAI: gpt-audio; Google: gemini-2.5-*)
await ai.chat({
  model: "gpt-audio",
  messages: [{ role: "user", content: [
    { type: "audio", source: { type: "base64", mediaType: "audio/wav", data: wavB64 } },
    { type: "text", text: "Transcribe this." },
  ] }],
  maxTokens: 256,
});

Agents: build, run, and test

A persona is a role as config-as-data: a model, a system prompt, its tools, and reasoning knobs. runAgent drives the tool-use loop (model turn → run tool handlers → feed results back → repeat) until the model stops or a step limit hits.

import { init, runAgent, type Persona, type ToolDef } from "@odla-ai/ai";

const ai = init({ keys: { anthropic: process.env.ANTHROPIC_API_KEY } });

const add: ToolDef = {
  name: "add",
  description: "Add two integers.",
  inputSchema: { type: "object", properties: { a: { type: "number" }, b: { type: "number" } }, required: ["a", "b"] },
  handler: (input) => ({ content: String((input.a as number) + (input.b as number)) }),
};

const calculator: Persona = {
  name: "calculator",
  model: "claude-opus-4-8",
  system: "Use the add tool for arithmetic, then state the result.",
  tools: [add],
  maxSteps: 4,
};

const run = await runAgent(ai, calculator, { input: "What is 21 + 21?" });
console.log(run.finalText);   // "…42"
console.log(run.toolCalls);   // [{ toolUse, output }]
console.log(run.usage);       // accumulated across turns

Provider calls and tool handlers receive a combined cancellation signal. Runs can also stop before additional effects when a run-wide budget is exhausted:

const run = await runAgent(ai, calculator, {
  input: "Reconcile these records",
  signal: request.signal,
  deadline: Date.now() + 30_000,
  budget: { maxInputTokens: 20_000, maxOutputTokens: 4_000, maxToolCalls: 8 },
});

Token limits are post-turn ceilings because provider input usage is known only after a response. maxInputTokens and maxTotalTokens therefore cannot tokenizer-bound the first prompt; they stop later turns and tool effects once reported usage reaches the ceiling. maxOutputTokens and the remaining total budget cap each request's maxTokens before dispatch. Limits must be integers (maxToolCalls may be zero; token limits must be positive). When a limit blocks tools, the loop records a matching error tool_result for every requested tool before persisting memory, so retries never inherit an unmatched effect request. stoppedReason is "budget_exhausted" when a limit prevents further work.

Tools can declare taint constraints (outputTaint / acceptsTaint) for a lightweight CaMeL-style prompt-injection gate, and personas can carry a pluggable MemoryScope.

Skills

A skill is a named bundle of tools + instructions. Attach skills to a persona and their tools are merged into the model's tool list and their instructions into the system prompt — the primitive for "add a skill and call it as tools."

import { entityCrudSkill, runAgent } from "@odla-ai/ai";
import { init as odlaInit } from "@odla-ai/db";

const db = odlaInit({ appId, adminToken: appKey, endpoint: ODLA_URL });

// Turn an odla-db entity into list/create/update/delete tools that change data.
const todos = entityCrudSkill({
  db,
  entity: "todos",
  fields: { text: { type: "string" }, done: { type: "boolean" } },
  required: ["text"],
});

const assistant = { name: "todo-bot", model: "claude-opus-4-8", skills: [todos] };
const run = await runAgent(ai, assistant, { input: "Add 'buy milk' and show my open todos." });
// the agent calls create_todo then list_todos, writing/reading via odla-db

entityCrudSkill generates list_<plural>, create_<singular>, update_<singular> (partial — pass only the fields that change, e.g. toggle done), and delete_<singular> tools whose handlers call db.transact/db.query. Or hand-roll a Skill = { name, instructions?, tools: ToolDef[] } for anything else.

Evals

import { evaluate, exactMatch, llmJudge } from "@odla-ai/ai";

const report = await evaluate({
  inference: ai,
  model: "claude-opus-4-8",
  system: "Answer with only the number.",
  grader: exactMatch(),
  cases: [
    { input: "What is 2+2?", expected: "4" },
    { input: "What is 10-3?", expected: "7" },
  ],
});
console.log(`${report.passed}/${report.total} passed`);

Graders: exactMatch, includes, structuredMatch, and llmJudge (which dogfoods the inference core to grade open-ended output against a rubric). Pass a persona instead of a model to evaluate a full agent.

Keys & roles

Keys are BYO — a static per-provider map and/or a dynamic resolveKey (multi-tenant / per-call). A provider with no resolvable key is simply unavailable: requesting one of its models throws a ConfigError.

const ai = init({
  resolveKey: (provider) => lookupTenantKey(currentTenant, provider),
});

"Roles" are personas (above) — config-as-data, not RBAC.

Storage, memory & BYO keys via odla-db

odla-ai follows odla-db's model: it's a self-contained npm package whose README and exported TypeScript declarations/JSDoc describe the installed API, and it integrates with @odla-ai/db for durable storage — without taking a runtime dependency on it. You install @odla-ai/db, build a client, and inject it; odla-ai talks to it through a structural interface. The rendered public reference is at https://odla.ai/docs/packages/ai.

The agent handshake (an agent never takes a platform secret):

import { requestToken, init as odlaInit } from "@odla-ai/db";
import { provisionAgentApp, odlaDbKeyResolver, OdlaDbMemory, persistRun, runAgent, init } from "@odla-ai/ai";

const ODLA_PLATFORM = "https://odla.ai";
const ODLA_DB_URL = "https://db.odla.ai";

// 1. Device authorization — the matching account signs in, reviews, and
//    approves the exact code; the agent receives a revocable platform token.
const { token } = await requestToken({
  endpoint: ODLA_PLATFORM,
  email: process.env.ODLA_USER_EMAIL!, // identity hint, not a password or token
  onCode: ({ userCode, verificationUriComplete }) => console.log(verificationUriComplete ?? userCode),
});

// 2. Provision: create app, mint an app key, push the agent schema, and store the
//    user's BYO provider keys as odla-db secrets (encrypted at rest).
const { appId, appKey } = await provisionAgentApp({
  endpoint: ODLA_PLATFORM,
  token,                                   // operator/dev token — needed to WRITE secrets
  providerKeys: { openai: OPENAI_KEY, anthropic: ANTHROPIC_KEY },
});

// 3. Runtime: read keys from secrets with the app key, persist memory + run traces.
const db = odlaInit({ appId, adminToken: appKey, endpoint: ODLA_DB_URL });
const ai = init({ resolveKey: odlaDbKeyResolver(db) });   // keys come from odla-db secrets

const persona = { name: "assistant", model: "gpt-5", memory: new OdlaDbMemory({ db, sessionId: "user-42" }) };
const run = await runAgent(ai, persona, { input: "Pick up where we left off." });
await persistRun(run, { db, sessionId: "user-42" });

BYO keys as secrets — the write/read split mirrors odla-db: storing a secret is a privileged action (operator or odla_dev_ token, via provisionAgentApp / putSecret); reading it at request time uses the ordinary read-only app key (odlaDbKeyResolver → db.secrets.get). Keys are AES-GCM-encrypted at rest and never returned to browser clients. OdlaDbMemory persists conversation turns and persistRun writes run traces, both as natural-key upserts (idempotent on retry).

Platform apps (odla.ai)

For apps registered on the odla platform, LLM access is an app setting: provider + default model live in the registry per environment (Studio's AI card, or @odla-ai/apps setAi), and the API key lives in the platform vault — the env's odla-db tenant secrets. initFromPlatform reads both:

import { initFromPlatform } from "@odla-ai/ai";
import { init as odlaInit } from "@odla-ai/db";

const db = odlaInit({ appId: TENANT, adminToken: env.ODLA_API_KEY, endpoint: "https://db.odla.ai" });
const { ai, provider, model } = await initFromPlatform({
  platform: "https://odla.ai",
  appId: APP_ID,          // the registry app id, not the tenant
  env: "prod",
  db,
});

The config comes from the anonymous public-config endpoint and is cached (~60s by default), so switching provider or model in Studio takes effect without a redeploy. The configured provider is enforced: its facade catalog is filtered to that provider, and a mismatched configured/default/call model fails with ConfigError. If Studio has no default model, each call must name a model for that provider. Tenant-bound facades are never shared globally. Provider keys and SDK clients have finite 60-second in-isolate caches; rotation replaces the previous credential generation rather than retaining it. Set keyResolverOptions: { ttlMs, cacheVersion } to shorten or immediately invalidate it after rotation. Never put provider keys in an app Worker's wrangler vars/secrets — the worker's only secret is its odla-db app key.

0.3 migration notes

• Tool/extraction JSON Schemas now fail closed. extract() throws ToolInputError; runAgent() records an error tool_result and does not run a local handler when its arguments do not validate.
• Platform calls cannot cross the provider selected in Studio.
• CancelledError (cancelled) and DeadlineExceededError (deadline_exceeded) distinguish caller limits from provider failures.
• Invalid run budgets now throw ConfigError; budget stops persist paired tool results and may therefore add one final user/tool-result turn to memory.

Models & capabilities

The catalog maps a canonical model id → provider, native id, and capabilities. The built-in DEFAULT_CATALOG covers current Anthropic, OpenAI, and Google models; extend or override it via buildCatalog / init({ catalog }).

| | image in | audio in | tools | thinking | effort | web search | structured | |---|---|---|---|---|---|---|---| | Claude (opus/sonnet/fable) | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | | Claude Haiku | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ✅ | | GPT-5 line | ✅ | ❌ | ✅ | — | ✅ | — | ✅ | | GPT-4o | ✅ | ❌ | ✅ | — | ❌ | — | ✅ | | gpt-audio | ❌ | ✅ | ✅ | — | ❌ | — | ✅ | | Gemini 2.5 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |

Model ids drift. The catalog is data — verify ids against each provider's current docs (the claude-api skill for Anthropic; /v1/models for OpenAI; Google GenAI docs for Gemini) rather than trusting them from memory.

Errors

Every error is an OdlaAIError subclass carrying a stable code — branch on err.code, not message text: ConfigError, AuthError, RateLimitError, CapabilityError, ContextWindowError, InvalidRequestError, ToolInputError, CancelledError, DeadlineExceededError, ProviderError.

OdlaAIError.usage is present only when a provider response was already received and a later forced-tool or schema check failed. It is a frozen, sanitized token-usage record so accounting can charge a failed structured response accurately; it may be absent for transport, authentication, timeout, or preflight failures. toShape() and streaming OracleErrorShape deliberately omit it.

Develop

npm run typecheck   # tsc --noEmit (strict)
npm test            # vitest — offline, mock-provider based
npm run build       # tsup → dual ESM + CJS + d.ts
npm run smoke       # live check against real providers (needs .dev.vars)

For the live smoke, copy .dev.vars.example → .dev.vars and fill in whatever keys you have; providers without a key are skipped.