@tokenmeter/sdk

In-process LLM usage capture for TypeScript/Node. Wraps a provider client you already have, reads token usage out of the response object it already returns (no extra network hop), and ships batches to the Token Meter Ingest API in the background.

Published to npm as @tokenmeter/sdk. The SDK is public and installable by anyone — it does nothing without a tl_live_ ingest key; rate limits, tier caps, and soft-lock are all enforced server-side at the ingest door.

Install

npm install @tokenmeter/sdk

Supported providers

| Provider | Wrapper | Notes | | --- | --- | --- | | OpenAI | wrapOpenAI | chat completions + Responses API | | Anthropic | wrapAnthropic | merges streamed usage deltas | | Google Gemini | wrapGemini | | | AWS Bedrock | wrapBedrock | Converse API + InvokeModel (Anthropic/Nova) | | Azure OpenAI | wrapAzureOpenAI | | | xAI, Mistral, Fireworks, Together AI, Vercel AI Gateway | wrapOpenAICompatible | OpenAI-compatible Chat Completions / Responses clients; retained as the upstream provider | | Vercel AI SDK | tokenmeterMiddleware | via wrapLanguageModel middleware, not a client wrapper |

Each provider returns usage in its own shape (cache tokens, reasoning tokens, batching discounts differ per vendor). The normalize*Usage functions in src/normalize/ convert every shape into one disjoint { input, output, cacheRead, cacheWrite, reasoning } schema before it's shipped — see docs/FRAMEWORKS.md for the full per-provider caveats (LangChain/LlamaIndex/LangGraph are not covered yet).

Quick start

import OpenAI from "openai";
import { TokenMeterClient, wrapOpenAI, action } from "@tokenmeter/sdk";

const tl = new TokenMeterClient({ apiKey: process.env.TOKENMETER_KEY! });
const openai = wrapOpenAI(new OpenAI(), tl);

await action("generate-report", { customerRef: "acme", feature: "reports" }, async () => {
  await openai.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: "hi" }],
  });
  // usage for this call is captured automatically, tagged with
  // customer_ref=acme, feature=reports, and a shared trace_id.
});

Anthropic is the same shape: wrapAnthropic(new Anthropic(), tl).

Optimization measurements for agents

Enable local prompt-prefix fingerprinting once with new TokenMeterClient({ apiKey, optimization: { promptCaching: true } }). Declare delay tolerance only for suitable background tasks with action("report", { optimization: { maxCompletionDelayMs: 86400000 } }, runAgent). The SDK includes metadata in its normal usage events; users do not add HTTP headers. Both settings are optional and default to off/absent. An inner action can disable them with promptCaching: false and maxCompletionDelayMs: null. See integration and deployment details for supported agents, privacy behavior and detector requirements.

Prompt topics

Label each call with what it is about, for Analytics > Use cases, without sending any prompt text:

import { TokenMeterClient, keywordTopicClassifier } from "@tokenmeter/sdk";
const tl = new TokenMeterClient({ apiKey, optimization: { topicClassifier: keywordTopicClassifier } });

The classifier runs in your process on the latest user message and returns a topic key (such as debugging or legal_review); only that key is sent, as tags.prompt_topic. Anything that is not a key (/^[a-z][a-z0-9_]{1,39}$/) is dropped. Supply your own synchronous function for better labels; an async one, or one that throws, is ignored and the call is untouched. Turn it off for one action with { optimization: { topicClassifier: null } }.

OpenAI-compatible providers

Use the provider-aware wrapper when an OpenAI client targets xAI, Mistral, Fireworks, Together AI, or Vercel AI Gateway ("vercel_gateway"). This preserves the actual provider on the metered event, which is required for its pricing and reconciliation records to match.

const together = wrapOpenAICompatible(
  new OpenAI({ apiKey: process.env.TOGETHER_API_KEY, baseURL: "https://api.together.xyz/v1" }),
  tl,
  "together",
);

Read usage data

Create a separate read-only key under Settings → API keys → Read API keys. It is not an ingest key and cannot submit usage events.

import { TokenMeterReadClient } from "@tokenmeter/sdk";

const readApi = new TokenMeterReadClient({
  apiKey: process.env.TOKEN_METER_READ_KEY!,
  baseUrl: process.env.TOKEN_METER_DASHBOARD_URL!,
});
const page = await readApi.listDailyUsage({ start: "2026-10-01", end: "2026-10-31" });

The endpoint returns daily metered usage/spend summaries, not provider invoice data. See docs/openapi.yaml for pagination, fields, and limits.

Context / attribution

action(name, attrs, fn) opens a named scope backed by AsyncLocalStorage — every wrapped provider call made inside fn (including across Promise.all fan-out and nested action() calls) is stamped with trace_id / parent_span_id / action / customer_ref / feature / team / environment. Nested calls inherit attrs from the enclosing scope unless overridden. For hops AsyncLocalStorage can't see across on its own (a queue, a worker thread), capture current() on one side and attach(ctx, fn) on the other.

Delivery model

Events go into an in-memory ring buffer (default 10k) and flush every 2s or 500 events, whichever's first. record() is synchronous and never throws or blocks a provider call. On buffer overflow the oldest events drop and the drop count ships in the next batch header — visible on the coverage screen, never silent. On process exit, beforeExit triggers a best-effort flush capped at 2s.

Serverless (Lambda / Vercel / Cloudflare)

The 2s background timer is wrong on any runtime that freezes the process between invocations — a frozen setTimeout just never fires. Use createServerlessClient instead, which disables it, and flush explicitly:

// Vercel / Cloudflare edge — extend execution past the response:
import { createServerlessClient, flushOnWaitUntil } from "@tokenmeter/sdk/serverless";
const tl = createServerlessClient({ apiKey: process.env.TOKENMETER_KEY! });
// ...handle request, wrapped calls happen...
flushOnWaitUntil(tl, ctx); // ctx.waitUntil(...)

// AWS Lambda — await it as the literal last line before return:
import { createServerlessClient, flushBeforeReturn } from "@tokenmeter/sdk/serverless";
const tl = createServerlessClient({ apiKey: process.env.TOKENMETER_KEY! });
export const handler = async (event: unknown) => {
  const result = await doWork(event);
  await flushBeforeReturn(tl);
  return result;
};

Loss bound: events recorded after the last flush of an invocation are lost if that invocation returns without calling flush()/waitUntil() — there is no background timer left to catch them once serverless mode is on. serverless.ts documents this in full.

Vercel AI SDK

wrap(client) never sees a client the AI SDK instantiates internally. Use the middleware instead:

import { wrapLanguageModel } from "ai";
import { openai } from "@ai-sdk/openai";
import { tokenmeterMiddleware } from "@tokenmeter/sdk";

const model = wrapLanguageModel({
  model: openai("gpt-4o-mini"),
  middleware: tokenmeterMiddleware(tl),
});

See docs/FRAMEWORKS.md for the honest support matrix (LangChain, LlamaIndex, LangGraph are not covered yet). Works with AI SDK v3 through v7; cache and reasoning tokens are split out on v5+ when the provider reports them (v3/v4 only expose prompt/completion totals).

Vercel AI Gateway

Models from the AI SDK's Gateway provider are metered as vercel_gateway with Vercel's creator/model id. To make Vercel's own tag report match your attribution, opt in to sending the call's team, project, feature and environment as Gateway reporting tags (Vercel bills each tag written, so this is off by default; AI SDK 5+ only):

import { gateway, wrapLanguageModel } from "ai";

const model = wrapLanguageModel({
  model: gateway("anthropic/claude-sonnet-4.5"),
  middleware: tokenmeterMiddleware(tl, { team: "payments" }, { gatewayReportingTags: true }),
});
// sends ai-reporting-tags: team:payments (plus project:/feature:/env: when set)

Tags you already send are kept and Vercel's 10-tag limit is respected; customer references are never sent.

Testing

npm test (tsx --test test/*.test.ts) — Node's built-in test runner, no added test framework. Fixtures for the disjoint-normalization CI invariant live in test/fixtures/.