@vovix/llm

TypeScript client for the Vovix LLM API — one API key to call any model on Amazon Bedrock Mantle: chat, schema-valid JSON, and tool-calling agents. No AWS SDK, no credentials, zero dependencies: it only needs fetch (Node.js 18+).

Documentation: https://llm.vovix.io/sdks

npm install @vovix/llm

import { createClient, VovixLlmError } from '@vovix/llm'

const llm = createClient({ baseUrl: 'https://api.llm.vovix.io/v1', apiKey: process.env.LLM_API_KEY! })

const { text, costUsd } = await llm.chat({
  model: 'google.gemma-3-12b-it',
  messages: [{ role: 'user', content: 'Say hello in one word.' }],
  maxTokens: 32,
})

• llm.chat(req) — one complete turn
• llm.stream(req, { onText }) — the same turn, text delivered as the model writes it
• llm.object<T>(req) — JSON matching a JSON Schema
• llm.runAgent(req) — a tool-calling agent: the SDK runs your tools and loops until the model answers; pass onText to stream every turn
• llm.agent(req) — a single agent turn, if you want to drive the loop yourself
• llm.usage() — your service's spend and tokens today
• llm.models() — the model catalog, with prices

import { defineTool } from '@vovix/llm'

const weather = defineTool<{ city: string }>({
  name: 'get_weather',
  description: 'Current weather for a city',
  schema: { type: 'object', properties: { city: { type: 'string' } }, required: ['city'] },
  run: async ({ city }) => fetchWeather(city),
})

const result = await llm.runAgent({
  model: 'openai.gpt-oss-120b',
  messages: [{ role: 'user', content: 'Should I bring an umbrella in Tokyo today?' }],
  maxTokens: 600,
  tools: [weather],
  maxTurns: 6, // model turns for the whole run
  maxUsd: 0.05, // stop once the run has spent this much
})
console.log(result.status, result.text) // 'done', "Yes — rain is expected…"

A tool that throws, or arguments that fail parse, don't end the run: the error is sent to the model as the tool result so it can correct itself.

Every failure is a VovixLlmError with a code to branch on; status is 0 when no HTTP response arrived (TIMEOUT, NETWORK). Gateway throttles and 5xx are retried; BUDGET_EXCEEDED is not.

An API key is issued per service by the Vovix admin. See Authentication, Models and Errors.

License

MIT