belvedir

The Belvedir SDK: auto-instrument your AI application for recursive self-improvement.

Building in Python? Use belvedir on PyPI instead: same concepts, Python idioms.

Already using @belvedir/loop? Nothing breaks: @belvedir/[email protected] stays on npm and keeps working. It just won't get new releases, so switch when convenient: change the dependency to belvedir and the import path. The API is identical.

Install

npm install belvedir

Quick Start (Next.js App Router)

Traceloop patches the Anthropic / OpenAI SDKs when you call initialize(). Two rules make this reliable in Next.js:

1. Call initialize() from instrumentation.ts: the only place guaranteed to run before your routes load.
2. Pass your LLM SDKs via instrumentModules: Next.js can bundle or ESM-load the SDKs in ways auto-instrumentation can't see; passing them explicitly patches them no matter how they were loaded.

Create instrumentation.ts at your project root (or src/instrumentation.ts if using a src/ folder):

// instrumentation.ts
export async function register() {
  if (process.env.NEXT_RUNTIME === "nodejs") {
    const { initialize } = await import("belvedir");
    // Only import the SDKs you actually use.
    const openai = await import("openai");
    const anthropic = await import("@anthropic-ai/sdk");
    initialize({
      apiKey: process.env.BELVEDIR_API_KEY!,
      appName: "my-ai-app",
      instrumentModules: { openAI: openai.OpenAI, anthropic },
    });
  }
}

Do not call initialize() from inside a route file, and only call it once.

In a plain Node.js process, call initialize() at the top of your entry point instead; instrumentModules is then optional as long as it runs before any LLM SDK import.

Sessions: link traces so Belvedir can find tasks

This is what powers tasks and task groups. Wrap each agent run in withSession so every LLM and tool span is linked under one session id. Optionally wrap distinct units of work in task() for sharper task boundaries:

import loop from "belvedir";

await loop.withSession({ sessionId: chatId, userId: user.id }, async () => {
  await loop.task("send_email", async () => {
    await agent.run("email danny the report");
  });
});

Without withSession you still get raw traces, but they can't be grouped into sessions or segmented into tasks.

Serverless (Vercel): flush before returning

Vercel functions can freeze or exit before the batching span processor exports. Call flush() at the end of any route handler that makes LLM calls:

import Anthropic from "@anthropic-ai/sdk";
import { flush } from "belvedir";

const anthropic = new Anthropic();

export async function POST(req: Request) {
  const { prompt } = await req.json();

  const message = await anthropic.messages.create({
    model: "claude-sonnet-4-6",
    max_tokens: 1024,
    messages: [{ role: "user", content: prompt }],
  });

  await flush();
  return Response.json(message);
}

If you set disableBatch: true in initialize(), every span is exported immediately and flush() isn't strictly required, but it's still cheap to call and is the safer default for production.

Report outcomes (ground truth)

When your app knows whether a session actually worked (the order shipped, the test passed, the user accepted), tell Belvedir. Reported outcomes beat the platform's model-judged labels: a "fail" keeps every task in that session out of training data.

import { flush, reportOutcome } from "belvedir";

await flush(); // the session must have arrived first
await reportOutcome(sessionId, orderShipped ? "success" : "fail");

reportOutcome never throws and returns whether the report was accepted (false with a warning if Belvedir hasn't seen the session yet; retry after flush(), or report later from a webhook or job).

Configuration

| Option | Type | Default | Description | |--------|------|---------|-------------| | apiKey | string | required | Your Belvedir API key (bv_live_...) | | baseUrl | string | https://platform.belvedir.ai | Belvedir platform URL | | appName | string | belvedir-loop-app | Your application name | | disableBatch | boolean | false | Disable span batching (export each span immediately; useful for local testing) | | instrumentModules | object | – | Your imported LLM SDKs, e.g. { openAI: openai.OpenAI, anthropic }. Patches them directly instead of relying on require hooks. Required in Next.js and other bundled/ESM environments. Either the SDK class or the module namespace is accepted. | | instrumentFetch | boolean | true | Capture LLM calls made with raw fetch to OpenAI-compatible endpoints (POST .../chat/completions), including streamed responses. Set false to opt out. |

API

| Function | Description | |----------|-------------| | initialize(config) | Patches your LLM clients and starts exporting spans. Call once, before LLM SDKs load (or pass them via instrumentModules). | | withSession(info, fn) | Runs fn with every produced span linked under the session. info is a session id string, or { sessionId, userId?, metadata? } (both forms work since 0.3.1). Required for tasks and task groups. | | task(name, fn) | Optional boundary hint: wraps a unit of agent work in a named span so segmentation knows where a task starts and ends. | | flush() | Forces pending spans to export. Await it before a serverless route returns. | | reportOutcome(sessionId, outcome) | Reports ground truth ("success" or "fail") for a session, after flush(). Never throws; returns whether the report was accepted. Also on the default export. |

Raw fetch and streaming (0.4.0)

Not every LLM call goes through the openai / @anthropic-ai/sdk clients: agent frameworks often ship their own HTTP layer, and apps call OpenAI-compatible endpoints (the Belvedir router, OpenRouter, vLLM, ...) with plain fetch. Since 0.4.0 initialize() also instruments globalThis.fetch: a POST to a .../chat/completions path with a JSON messages body becomes a full LLM span: prompt from the request, completion from the response. Streamed (SSE) responses are captured too: the response is cloned, deltas are accumulated in the background while your app reads its own stream, and the final assistant message, tool calls, finish reason, and usage land on the span when the stream ends. Other requests pass through untouched, and capture errors can never break your fetch. Opt out with instrumentFetch: false.

AI SDK apps (the ai package)

The raw-fetch path is also how AI SDK apps are captured: make the calls with @ai-sdk/openai-compatible (pointed at your provider, your gateway's OpenAI-compatible endpoint, or Belvedir's router) and every generateText/streamText call is traced automatically; no instrumentModules entry needed, just call initialize() first. The AI SDK's native provider packages (@ai-sdk/openai, @ai-sdk/anthropic) and plain "provider/model" gateway strings use protocols this SDK doesn't instrument and produce no LLM spans. Full setup: https://docs.belvedir.ai/guides/ai-sdk

How It Works

1. Instrument: initialize() patches LLM client prototypes so every call becomes an OpenTelemetry span with prompt, completion, model, and token usage attributes.
2. Collect: Spans are batched and POSTed to ${baseUrl}/api/v1/traces.
3. Analyze: Belvedir's analysis models study trace patterns against your improvement target.
4. Improve: Belvedir creates GitHub PRs with suggested code changes.

Supported Providers

Everything Traceloop's Node instrumentations cover: OpenAI (Azure OpenAI included, via the same openai client), Anthropic, Amazon Bedrock, Google Vertex AI, Google Generative AI, Cohere, Together, the LangChain and LlamaIndex frameworks, MCP, and the Chroma, Pinecone, and Qdrant vector stores. Any OpenAI-compatible endpoint called with the openai client or raw fetch is traced too (see above). Replicate and HuggingFace are covered by the Python SDK only.

Local Models (Ollama)

Agents running against local models are traced the same way as cloud providers. Ollama (or the Local Inference desktop app, which bundles it) exposes an OpenAI-compatible API, so point the OpenAI SDK at it and the instrumentation above just works:

const client = new OpenAI({
  baseURL: "http://127.0.0.1:11434/v1",
  apiKey: "ollama", // required by the SDK, ignored by Ollama
});

127.0.0.1:11434 is Ollama's default bind address; if you've set OLLAMA_HOST or run Ollama in Docker with a remapped port, use that address in baseURL instead.

Initialize with instrumentModules: { openAI: openai.OpenAI } and wrap runs in withSession as usual. Note that while inference stays on your machine, exported traces (prompts and completions) are sent to Belvedir.

The same pattern covers hosted open models (Together, Fireworks, Groq, OpenRouter, or self-hosted vLLM; anything OpenAI-compatible): use the provider's base URL and a real API key. You can also mix models, e.g. a small local model for cheap steps and a bigger hosted one for hard reasoning. Multiple OpenAI client instances share the one instrumented class, and each span records the model that served it.

Local Development

If running Belvedir locally:

initialize({
  apiKey: "your-key",
  baseUrl: "http://localhost:3000",
  appName: "my-app",
  disableBatch: true, // Send spans immediately
});

Troubleshooting

I see HTTP spans but no anthropic.chat / openai.chat span. The patch didn't take. Two things must both be true in Next.js:

1. Pass the SDKs via instrumentModules (see Quick Start): the bundler can load LLM SDKs in ways require() hooks can't see, so auto-instrumentation misses them.
2. Keep the SDKs external so every bundle shares one runtime copy. If webpack inlines them, each bundle gets a private copy and instrumentModules patches one your routes never use. List belvedir itself as well: a route or server action that bundles its own copy calls withSession()/flush() on an uninitialized duplicate, so flush silently no-ops and spans drop on serverless. The config key is version-specific: serverExternalPackages on Next 15+, experimental.serverComponentsExternalPackages on Next 13/14. The wrong name is silently ignored; watch for the Unrecognized key(s) build warning.

In plain Node, also make sure initialize() runs before any LLM SDK import; the [belvedir/loop] initialize() ran after ... was imported startup warning means it didn't.

Spans appear locally but not on Vercel. The function exited before the batch flushed. Call await flush() before your route returns.

The process died at startup with an unhandled rejection mentioning api.traceloop.com. The underlying Traceloop SDK's "sync" feature (prompt-config fetching from Traceloop's servers) defaults on, and a network failure there rejects an unhandled promise. Versions after 0.4.1 turn it off inside initialize(); on 0.4.1 or earlier set TRACELOOP_SYNC_ENABLED=false in the environment. Belvedir never uses that feature and no data goes to Traceloop either way.

Traces arrive but no tasks or task groups form. Work must run inside withSession. Tasks appear ~30s after a session goes quiet.