rouva

Official Node.js SDK for Rouva — managed AI gateway with intelligent routing and spend tracking.

Installation

npm install @rouvanpm/rouva

Quick Start

import { Rouva } from '@rouvanpm/rouva'

const rouva = new Rouva({ apiKey: 'rva_...' })

const response = await rouva.chat.completions.create({
  messages: [{ role: 'user', content: 'Summarize the benefits of AI routing.' }],
})

console.log(response.choices[0].message.content)

Provider agnostic

Rouva works with all connected providers — Anthropic, OpenAI, Gemini, DeepSeek, Mistral, Moonshot, xAI, and Z.ai. You can request a specific model, force a specific provider, or omit both and let Rouva route to the cheapest capable model automatically.

// Request a specific model
const res = await rouva.chat.completions.create({
  model: 'gpt-4o',
  messages,
})

// Force a specific provider + model
const res = await rouva.chat.completions.create({
  provider: 'gemini',
  model: 'gemini-2.5-pro',
  messages,
})

// Let Rouva decide — routes to cheapest model for the task
const res = await rouva.chat.completions.create({
  messages,
})

OpenAI-style request shape

// Before
import OpenAI from 'openai'
const openai = new OpenAI({ apiKey: '...' })
const res = await openai.chat.completions.create({ messages, model: 'gpt-4o' })

// After — Rouva routes to the cheapest capable model automatically
import { Rouva } from '@rouvanpm/rouva'
const rouva = new Rouva({ apiKey: 'rva_...' })
const res = await rouva.chat.completions.create({ messages })

Streaming

const stream = await rouva.chat.completions.create({
  messages: [{ role: 'user', content: 'Write a short story.' }],
  stream: true,
})

const reader = (stream as ReadableStream).getReader()
const decoder = new TextDecoder()

while (true) {
  const { done, value } = await reader.read()
  if (done) break
  process.stdout.write(decoder.decode(value))
}

Streaming responses are normalized to OpenAI-style SSE chunks, even when Rouva routes the request to Anthropic.

stream is a client-side toggle: stream: true returns the raw ReadableStream, omitting it (or stream: false) returns a buffered ChatCompletion. It is never sent to the gateway.

Request parameters

const res = await rouva.chat.completions.create({
  model: 'gpt-4o-mini',                    // required when using seed (OpenAI only)
  messages: [{ role: 'user', content: 'Write a haiku about routing.' }],
  system: 'You are a concise assistant.',  // string or Anthropic-style text blocks
  max_tokens: 1024,                        // default 4096
  temperature: 0.7,                        // 0–1
  top_p: 0.9,                              // nucleus sampling, (0, 1]
  stop: ['END'],                           // up to 4 stop sequences
  seed: 42,                                // pins the request to this exact model
})

• temperature — sampling temperature between 0 and 1. Reasoning models (the gpt-5 family) only support their default temperature, so the gateway omits it when routing to one.
• max_tokens — values below 1024 also steer auto-routing away from reasoning models, which would otherwise spend the whole budget on hidden reasoning tokens.
• top_p — nucleus sampling, forwarded to every provider; top_p: 1 is treated as omitted. OpenAI reasoning models don't support it: auto-routing avoids them when it's set, and pinning one returns a 400.
• stop — a string or up to 4 stop sequences, forwarded to every provider (Anthropic receives them as stop_sequences).
• seed — best-effort deterministic sampling. Only OpenAI honors it, so it requires an OpenAI model and pins the request to that exact model (no routing, no fallbacks).

Unsupported OpenAI options (response_format, logit_bias, n > 1, the legacy functions/function_call fields, …) are rejected by the gateway with an explicit 400 rather than silently ignored.

Unlisted models

Dated snapshots (gpt-4o-mini-2024-07-18) and fine-tunes (ft:gpt-4o-mini:…) work when pinned — they are matched to their provider by naming convention and forwarded as-is. Until the gateway's pricing registry knows them, the dashboard records their token counts with zero cost.

Models not recognised by the gateway return a 400 immediately:

{ "error": "Model \"<id>\" is not supported. See rouva.io/docs for the supported model list." }

OpenAI-compatible endpoint

Using the OpenAI SDK (or LangChain, the Vercel AI SDK, …) instead of this one? The gateway is also served at POST /v1/chat/completions — point baseURL at https://app.rouva.io/v1 with your rva_ key and it behaves exactly like OpenAI: model is required, responses are buffered JSON unless stream: true, and only OpenAI-format providers are available (use this SDK or the native endpoint for Anthropic models).

Non-tools requests on /v1 always honor the named model exactly. Tools requests on /v1 benefit from within-provider intelligent routing when Intelligent Routing is enabled in your dashboard — Rouva picks the cheapest capable model within the same provider, using the named model as a cost ceiling. The provider never changes (tool schemas are provider-specific), and response.model reflects the model that actually served the request. Turn Intelligent Routing off in dashboard Settings → Gateway to always pin the exact model.

Tool use

Tools are forwarded to your target provider verbatim — define them in the provider's own format (OpenAI { type: "function", function: {...} } or Anthropic { name, description, input_schema }) and pin the matching model. Tool schemas are provider-specific, so model is required and the gateway returns a 400 without it.

When using the /v1 endpoint (OpenAI SDK, LangChain, Vercel AI SDK, …) with Intelligent Routing enabled, tools requests are routed to the cheapest capable model within the same provider — see OpenAI-compatible endpoint for details. On the native SDK endpoint (/api/gateway/messages), the named model is always honored exactly.

const res = await rouva.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'What is the weather in SF?' }],
  tools: [{
    type: 'function',
    function: {
      name: 'get_weather',
      description: 'Get current weather for a city',
      parameters: {
        type: 'object',
        properties: { city: { type: 'string' } },
        required: ['city'],
      },
    },
  }],
})

const toolCall = res.choices[0].message.tool_calls?.[0]
if (toolCall) {
  const args = JSON.parse(toolCall.function.arguments)
  const weather = await getWeather(args.city)

  // Send the result back — same messages array plus the tool turn
  const followUp = await rouva.chat.completions.create({
    model: 'gpt-4o',
    messages: [
      { role: 'user', content: 'What is the weather in SF?' },
      res.choices[0].message,
      { role: 'tool', content: JSON.stringify(weather), tool_call_id: toolCall.id },
    ],
    tools: [/* same tools every turn */],
  })
}

Responses are normalized to the OpenAI shape regardless of provider: Anthropic tool_use blocks arrive as message.tool_calls (buffered) or delta.tool_calls chunks (streaming), with Anthropic's stop_reason: "tool_use" mapped to finish_reason: "tool_calls". When sending Anthropic results back, use the Anthropic dialect in your messages (tool_result content blocks) — message payloads pass through to the provider verbatim.

Don't branch on finish_reason to detect tool calls — check for the presence of message.tool_calls instead. finish_reason follows each provider's own semantics, and OpenAI notably returns "stop" (not "tool_calls") when tool_choice forces a specific function. The SDK passes OpenAI's values through unchanged so behavior matches calling OpenAI directly.

Tools requests record usage and cost. Savings are recorded when within-provider routing substitutes a cheaper model on /v1. Tools responses are not quality-scored or served from the semantic cache.

Cost optimisations

Rouva applies several cost-saving layers automatically on every non-tools request — no configuration required.

Semantic cache (A) — repeated or semantically similar prompts are served from cache without hitting the upstream provider. Cache hits record zero cost and appear in your dashboard with a semantic_cache_hit flag.

Prompt caching (B) — Anthropic requests automatically include cache-control headers on large system prompts and conversation prefixes, reducing input token costs on repeated turns.

Conversation summarisation (C2) — long Anthropic conversations are summarised by a cheap model before forwarding, compressing the context window and reducing input tokens. The summarisation cost is included in the request's spend snapshot. Only runs when the conversation exceeds ~2,000 tokens of non-system content and the request is not a tools request.

All three are skipped for tools requests, which are forwarded verbatim.

Options

const rouva = new Rouva({
  apiKey: 'rva_...',        // Required — get this from your Rouva dashboard
  baseURL: 'https://...',   // Optional — override the gateway URL
})

Agentic runs — rouva.run()

rouva.run() hands the entire agent loop to the Rouva gateway. You define your tools and handler URLs; Rouva calls the model, dispatches tool calls in parallel to your endpoints, and loops until the model stops or a limit is reached — no client-side loop, no streaming required.

const result = await rouva.run({
  model: 'claude-haiku-4-5-20251001',
  messages: [{ role: 'user', content: 'What is the weather in Paris and London?' }],
  tools: [
    {
      name: 'get_weather',
      description: 'Get current weather for a city',
      input_schema: {
        type: 'object',
        properties: { city: { type: 'string' } },
        required: ['city'],
      },
    },
  ],
  tool_handlers: {
    get_weather: 'https://api.example.com/weather',  // gateway POSTs tool input here
  },
  max_turns: 5,
  session_budget_usd: 0.10,  // optional hard cost cap
})

console.log(result.content)           // final model response
console.log(result.turns)             // number of turns taken
console.log(result.tool_calls_made)   // total tool calls dispatched
console.log(result.session_id)        // dashboard session ID
console.log(result.session_cost_usd)  // total USD spent

How it works

1. The gateway sends your messages to the model (non-streaming)
2. When the model returns tool calls, all handlers are dispatched in parallel via POST — the request body is the tool's JSON input, the response body is used as the tool result
3. Results are appended and the loop continues
4. The loop exits when finish_reason is "stop" / "end_turn", max_turns is reached, or session_budget_usd is exceeded

Every turn is logged as a usage snapshot in the Rouva dashboard under the shared session_id, giving you a full session replay with per-turn cost, model used, tool calls, and quality score.

Handler URL requirements

• Must be a public HTTPS URL (private/local addresses are rejected)
• The gateway sends a POST with Content-Type: application/json and the tool's input object as the body
• The response body (text or JSON) is returned to the model as the tool result

Parameters

| Parameter | Type | Required | Description | |---|---|---|---| | messages | ChatMessage[] | ✓ | Conversation seed — usually a single user message | | tools | object[] | ✓ | Tool definitions (OpenAI or Anthropic format) | | tool_handlers | Record<string, string> | ✓ | Tool name → handler URL mapping | | model | RouvaModel | ✓ | Model to use — required for all /run requests | | provider | RouvaProvider | | Force a specific provider | | max_turns | number | | Max loop iterations (default: 10) | | max_tokens | number | | Max tokens per turn (default: 4096) | | temperature | number | | Sampling temperature 0–1 | | session_budget_usd | number | | Hard USD cap across all turns |

Result shape

interface RunResult {
  content: string | null        // final model response text
  turns: number                 // turns executed
  tool_calls_made: number       // total tool calls dispatched
  session_id: string            // groups all turns in the dashboard
  finish_reason: string | null  // "stop", "end_turn", "max_turns", "budget_exceeded", "length"
  cost_usd: number              // cost of the last turn
  session_cost_usd: number      // cumulative cost across all turns
}

When to use run() vs chat.completions.create()

| | rouva.run() | chat.completions.create() | |---|---|---| | Tool loop | Gateway-managed | Client-managed | | Session grouping | Automatic | Manual (startSession) | | Streaming | No | Yes | | Use when | Fully delegating an agentic task | Building your own loop or need streaming |

Agent session tracking

Group all turns of an agent run under a single session ID so the Rouva dashboard can show per-session cost, token usage, and quality. Use this with chat.completions.create() when you manage the loop yourself.

const rouva = new Rouva({ apiKey: 'rva_...' })

// Start a session — auto-generates an ID and attaches it to every request
const sessionId = rouva.startSession()

// All turns carry the same session ID automatically
await rouva.chat.completions.create({ messages: [...], tools: [...] })
await rouva.chat.completions.create({ messages: [...], tools: [...] })
await rouva.chat.completions.create({ messages: [...], tools: [...] })

// End the session when the agent run is complete
rouva.endSession()

You can also read the active session ID at any time:

console.log(rouva.sessionId) // 'rva-sess-abc123' or undefined

Sessions are optional — requests without an active session are tracked individually as before.

Response metadata

Parsed non-stream responses may include a _rouva field with gateway header metadata when available:

const res = await rouva.chat.completions.create({ messages })

console.log(res._rouva)
// {
//   model_used: 'gpt-4o-mini',
//   provider_used: 'openai',
//   task_type: 'summarize'
// }

Getting your API key

1. Sign in to app.rouva.io
2. Go to Settings → Gateway Key
3. Generate a key — it starts with rva_

License

MIT