@llmtracer/sdk

Track cost, latency, and token usage across OpenAI, Anthropic, and Google Gemini — in one line of code.

Install

npm install @llmtracer/sdk

Quick Start

import llmtracer from '@llmtracer/sdk';

llmtracer.init({ apiKey: 'lt_...' });

// All OpenAI, Anthropic, and Google Gemini calls are now tracked automatically.

No wrappers, no manual instrumentation, no code changes. The SDK auto-patches your provider clients at import time.

View your dashboard at llmtracer.dev.

What Gets Captured

Every LLM call is automatically tracked with:

• Provider, model, tokens (input + output), latency, cost
• Google Gemini: thinking tokens (2.5 models), tool tokens, cached tokens
• Anthropic: cache creation + read tokens
• OpenAI: reasoning tokens (o1/o3/o4), cached tokens
• Caller file, function, and line number
• Auto-flush on process exit via process.on('beforeExit'), SIGINT, and SIGTERM

Environment Variable Pattern

import llmtracer from '@llmtracer/sdk';

llmtracer.init({
  apiKey: process.env.LLMTRACER_API_KEY,
  debug: true, // prints token counts to console
});

Multi-App Tracking

If you have multiple services sharing an API key, set appName to filter by application in the dashboard:

llmtracer.init({ apiKey: 'lt_...', appName: 'billing-service' });

Or via environment variable:

export LLMTRACER_APP_NAME=billing-service

Trace Context and Tags

await llmtracer.trace({ tags: { feature: 'chat', user_id: 'u_sarah' } }, async () => {
  const response = await openai.chat.completions.create({ ... });
});

Tags appear in the dashboard's Breakdown page and Top Tags card. Use them to answer questions like "which user costs the most?" or "which feature should I optimize?"

Tagging Patterns

| Pattern | Tag | Example | |---------|-----|---------| | Track cost by feature | feature | "chat", "search", "summarize" | | Track cost by user | user_id | "u_sarah", "u_mike" | | Track cost by customer (B2B) | customer | "acme-corp", "initech" | | Track cost by conversation | conversation_id | "conv_abc123" | | Track environment | env | "production", "staging" |

Supported Providers

| Provider | Package | Auto-patched | |----------|---------|-------------| | OpenAI | openai | ✅ | | Anthropic | @anthropic-ai/sdk | ✅ | | Google Gemini | @google/genai | ✅ |

Flushing Events

The SDK batches events and sends them in the background. In long-running processes (servers, daemons), this is fully automatic. For short-lived scripts and serverless environments, flush before the process exits.

Auto-flush (long-running processes)

By default the SDK registers handlers for process.on('beforeExit'), SIGINT, and SIGTERM:

import llmtracer from '@llmtracer/sdk';

llmtracer.init({ apiKey: 'lt_...' });

// Events are flushed automatically when the process exits

Manual flush (serverless / short-lived)

Call await llmtracer.flush() before returning from a handler or Lambda function:

import llmtracer from '@llmtracer/sdk';

llmtracer.init({ apiKey: 'lt_...', skipExitHandlers: true });

export async function handler(event) {
  const response = await openai.chat.completions.create({ ... });
  await llmtracer.flush(); // send before function returns
  return response;
}

SIGTERM handler (Cloud Run / Kubernetes)

process.on('SIGTERM', async () => {
  await llmtracer.flush();
  process.exit(0);
});

Debug Mode

Enable debug: true to print token counts to the console:

llmtracer.init({ apiKey: 'lt_...', debug: true });

[llmtracer] openai gpt-4o | 1,247 in → 384 out | $0.0094 | 1.2s
[llmtracer] anthropic claude-sonnet-4-5 | 2,100 in → 512 out (cache_read: 1,800) | $0.0031 | 0.8s
[llmtracer] google gemini-2.5-pro | 900 in → 280 out (thinking: 1,420) | $0.0067 | 2.1s

Streaming Support

Streaming calls are instrumented automatically. Token counts are captured from the final chunk:

const stream = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'Hello' }],
  stream: true,
});

for await (const chunk of stream) {
  // use chunk as normal
}

Configuration

| Option | Type | Default | Range | Description | |---|---|---|---|---| | apiKey | string | required | — | Your LLM Tracer API key (starts with lt_) | | appName | string | undefined | — | Application name for multi-app filtering. Falls back to LLMTRACER_APP_NAME env var | | endpoint | string | Production URL | — | Ingestion endpoint URL | | maxBatchSize | number | 50 | 1–500 | Max events per HTTP request | | flushIntervalMs | number | 5000 | 1000–60000 | Auto-flush interval in milliseconds | | maxQueueSize | number | 1000 | 100–10000 | Max events in queue before dropping oldest | | maxRetries | number | 3 | 0–10 | Max retry attempts for failed flushes | | sampleRate | number | 1.0 | 0.0–1.0 | Sampling rate. 0.5 captures ~50% of events | | debug | boolean | false | — | Enable debug logging to console | | skipExitHandlers | boolean | false | — | Skip process exit handlers (for plugins/serverless) |

All numeric options are validated on init(). Out-of-range values are replaced with the default, and a warning is logged when debug: true.

Reliability

The SDK is designed to never interfere with your application:

• Never throws — all internal errors are swallowed silently (enable debug: true for visibility)
• Batching — events are queued and sent in batches of maxBatchSize
• Retry with backoff — failed flushes are retried up to maxRetries times with exponential backoff (min(1000 * 2^attempt, 30000)) plus random jitter (0–1000ms)
• Drop after retries — after maxRetries consecutive failures, the batch is dropped to prevent unbounded memory growth
• Queue overflow — drops oldest events when the queue exceeds maxQueueSize
• Sampling — set sampleRate below 1.0 to reduce volume in high-throughput environments

License

MIT