kalmia-sdk

Automatic LLM call tracing for OpenAI and Anthropic. Wrap your client once and every call — prompts, responses, tool calls, errors — is captured and sent to your Kalmia dashboard.

Installation

npm install kalmia-sdk

openai and @anthropic-ai/sdk are optional peer dependencies — install whichever you use.

Usage

Get your API key from the Kalmia dashboard, then initialize the logger once at startup and wrap your LLM client:

import { initLogger, wrapAnthropic } from 'kalmia-sdk';
import Anthropic from '@anthropic-ai/sdk';

initLogger({
  projectName: 'my-agent',
  apiKey: 'kal_live_your_api_key', // required — or set KALMIA_API_KEY
});

const client = wrapAnthropic(new Anthropic());
// use `client` exactly like the Anthropic SDK — traces are captured automatically

OpenAI works the same way:

import { initLogger, wrapOpenAI } from 'kalmia-sdk';
import OpenAI from 'openai';

initLogger({ projectName: 'my-agent', apiKey: 'kal_live_your_api_key' });
const client = wrapOpenAI(new OpenAI());

Configuration

| Option / env var | Purpose | | --- | --- | | projectName (required) | Groups traces under a project. | | apiKey / KALMIA_API_KEY | Workspace API key. Without it, traces are rejected and dropped. | | baseUrl / KALMIA_BASE_URL | Where traces are sent. Defaults to the hosted Kalmia backend (https://www.kalmia.dev). Set this for local development or self-hosting. | | metadata | Optional key/value object merged into every trace's metadata (e.g. environment, version). |

Custom spans

Use traced to capture custom steps and currentSpan() to log inside them. Unlike the Python SDK's decorator, the TypeScript traced() wraps a callback (JavaScript has no decorator equivalent for arbitrary functions) — wrap the body of the function you'd otherwise decorate:

import { traced, currentSpan } from 'kalmia-sdk';

await traced(async () => {
  currentSpan().log({ input: { message } });
  // ...
}, { name: 'run' });

Tool spans ({ type: 'tool' }) are priced automatically when the logged output is the raw response of a recognized paid service (Exa, Tavily, Brave, Serper, and more). For any other paid call, include cost_usd in the logged output and Kalmia records it as that tool's cost:

await span.traced('scrape_page', async (toolSpan) => {
  const result = await client.scrape(url);
  toolSpan.log({ input: { url }, output: { ...result, cost_usd: 0.002 } });
}, { type: 'tool' });

Streaming

Streaming responses are captured automatically — the wrapped client accumulates the streamed content and usage and emits the span when the stream finishes (or when you break out of it early). Consume the stream exactly as you would the underlying SDK:

const stream = await client.chat.completions.create({ stream: true, /* ... */ });
for await (const chunk of stream) {
  // ...
}

Live trace streaming (opt-in)

By default the SDK ships a trace in a single post when its root traced call ends. With live trace streaming enabled, completed spans are also posted in mid-run batches while the run is still going, so long runs appear in the dashboard as they happen instead of only after they finish.

initLogger({
  projectName: 'my-agent',
  streaming: true,             // or KALMIA_STREAMING=1
  streamingBatchSize: 20,      // optional, this is the default
  streamingFlushIntervalMs: 5000, // optional, this is the default
});

| Option | Env var | Default | Purpose | | --- | --- | --- | --- | | streaming | KALMIA_STREAMING | false | Turn mid-run batch posting on. Off, the SDK behaves exactly like previous releases (one post at root end). | | streamingBatchSize | KALMIA_STREAMING_BATCH_SIZE | 20 | Flush a mid-run batch once this many completed-but-unsent spans have accumulated. | | streamingFlushIntervalMs | KALMIA_STREAMING_FLUSH_INTERVAL_MS | 5000 | Flush a smaller batch once this much time has passed since the last flush, checked at span end. |

How a streamed run stays coherent:

• Every mid-run batch reuses the trace id fixed at root creation and includes the root span without an end time, so the server keeps the run in progress and merges later batches cleanly.
• Mid-run batches are best-effort: a single send attempt, no retries, no buffering. A failed batch simply leaves its spans unsent.
• The final flush at root end is durable: it posts the complete root plus every span not already delivered, with the full retry-and-buffer contract from "Delivery reliability" below. A run whose mid-run batches all failed still arrives intact.
• Streaming never throws into your code and never blocks your spans; batches are posted in the background.

Session end times (Vercel AI SDK)

Session traces built through wrapAISDK({ sessionID }) follow the same live semantics, whether or not streaming is enabled:

• While a session is open, every post carries the session root without an end time, so the server keeps the run in progress and Kalmia's live analysis follows the session as it grows.
• endAISDKSession(sessionID) posts the root complete (with its end time), which moves the run to its terminal state and triggers the post-run analysis (judge, embedding) exactly once, over the whole session.
• A session that is never explicitly ended still goes terminal via the server's stall timeout once it stops receiving calls; calling endAISDKSession just makes that prompt.

Vercel AI SDK

To trace models used through the Vercel AI SDK, wrap them with wrapAISDK as LanguageModelV2 middleware:

import { wrapLanguageModel } from 'ai';
import { initLogger, wrapAISDK, endAISDKSession } from 'kalmia-sdk';

initLogger({ projectName: 'my-agent', apiKey: 'kal_live_your_api_key' });

const model = wrapLanguageModel({
  model: baseModel,
  middleware: [wrapAISDK({ sessionID })], // omit sessionID for one trace per call
});

Pass a sessionID in the metadata to group all of a session's calls into one trace. While the session is open, its trace stays in progress on the server; call await endAISDKSession(sessionID) when the session finishes to post the completed trace and release its root span (see "Session end times" above). A session you never end goes terminal via the server's stall timeout instead, and very long sessions are auto-evicted into a fresh trace segment. Tracing is observation-only: every streamed part is forwarded untouched, so it can never alter or break your model output.

Customer feedback

Relay your end customer's reaction to a run - a thumbs rating and/or an open-text comment - and Kalmia attaches it to the run's trace. Capture the trace id during the run with currentTraceId() (or span.traceId), store it with your own session record, then submit feedback whenever it arrives:

import { traced, currentTraceId, submitFeedback } from 'kalmia-sdk';

// During the run: capture the trace id and store it with your session.
let traceId: string | undefined;
await traced(async () => {
  traceId = currentTraceId();
  // ... run your agent, persist traceId with the session record ...
}, { name: 'my-agent-run' });

// Later, when the customer reacts:
const result = await submitFeedback(traceId, { rating: 'down', comment: 'Wrong date' });
if (!result.ok) console.warn('feedback not delivered:', result.error);

rating is 'up' or 'down'; comment is open text. Provided fields overwrite the trace's stored feedback state and omitted fields keep it - one current state per trace, no history. The server accepts feedback for a trace that has not finished ingesting, so no wait-for-trace logic is needed.

Invalid usage (no trace id, empty feedback, unknown rating) throws immediately. Delivery problems (network, auth, server errors) never throw: they come back as { ok: false, error } after bounded retry, so a feedback relay inside your own request handler cannot crash your API.

Delivery reliability

Trace delivery is at-least-once with no silent loss:

• Retry with backoff. Transient failures (network errors, timeouts, 429, 5xx) are retried with bounded exponential backoff. Permanent 4xx responses (400, 401, 413) are not retried.
• Buffering. A trace that still fails after its retries is held in a bounded in-memory buffer and re-attempted on the next send, then once more when the process exits.
• No silent loss. Every drop path — a non-retryable response, a full buffer evicting its oldest entry, a server-side rejection, or a trace still undelivered at exit — emits a console.warn naming the trace id, so a lost run is always visible in your logs.
• Safe retries. The backend de-duplicates by trace id, so a retried delivery never creates a duplicate.

The buffer lives in memory, so a hard process kill (SIGKILL, OOM, or a serverless freeze) can still lose buffered traces — those are warned when the trace is first buffered. Durable on-disk buffering is planned. Delivery never throws into your application: a failing dashboard never breaks your agent.

License

MIT