llm-usage-ts

Small TypeScript SDK that wraps the Vercel AI SDK to automatically persist LLM call metadata, token usage, and estimated costs to a local SQLite database. Comes with a minimal dashboard to explore calls, costs, and latency.

Highlights

• Drop-in wrapper around generateText, streamText, generateObject, streamObject
• SQLite persistence of prompts/messages, outputs, usage, latency, errors, tags
• Cost estimation via configurable per-token price table (OpenAI + Gemini included)
• Simple observability dashboard (Express + EJS) to browse calls and metrics
• Tagging and redaction support to help organize and protect sensitive data

Installation

npm install llm-usage-ts
# You bring your own provider adapters. For example:
npm install @ai-sdk/openai @ai-sdk/google

Requirements:

• Node 18+
• SQLite DB file is created automatically at ./.db/llm-usage.sqlite by default (configurable)

Note on imports: until the package exposes a stable top-level API, you can import directly from compiled files, e.g. llm-usage-ts/dist/llm.js.

Quick start (programmatic usage)

Example with OpenAI (via @ai-sdk/openai). The SDK works with any provider supported by the Vercel AI SDK.

import { createOpenAI } from '@ai-sdk/openai';
import { z } from 'zod';
// Deep import for now:
import { LLM } from 'llm-usage-ts/dist/llm.js';

const openai = createOpenAI({ apiKey: process.env.OPENAI_API_KEY! });

const llm = new LLM({
  dbDir: './.db',
  // Optional: redact before persisting
  // redact: (s) => s.replace(/[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+/g, '[REDACTED_EMAIL]'),
});

// 1) Text generation
const { text } = await llm.generateText(
  {
    model: openai('gpt-4o-2024-08-06'),
    prompt: 'Write a short product teaser for a note-taking app.'
  },
  { tags: ['teaser', 'marketing'] }
);
console.log(text);

// 2) Streaming text
const stream = await llm.streamText(
  {
    model: openai('gpt-4o-2024-08-06'),
    prompt: 'Stream a 3-sentence story about the ocean.'
  },
  { tags: ['demo'] }
);
for await (const chunk of stream.textStream) {
  process.stdout.write(chunk);
}

// 3) Structured output with Zod
const OutlineSchema = z.object({ title: z.string(), bullets: z.array(z.string()) });
const objRes = await llm.generateObject(
  {
    model: openai('gpt-4o-mini-2024-07-18'),
    schema: OutlineSchema,
    prompt: 'Make a short outline for a blog post about caching.'
  } as any,
  { tags: ['outline'] }
);
console.log(objRes.object);

The SDK automatically records each call in SQLite (with token usage, latency, status, tags, and output). Costs are computed from the configured price table.

Dashboard

Explore recent calls, filter by provider/model/tag, and view aggregate metrics.

Option A: Programmatic server

import { UsageDB } from 'llm-usage-ts/dist/db.js';
import { makeServer } from 'llm-usage-ts/dist/server.js';

const db = new UsageDB('./.db');
const app = makeServer(db);
app.listen(5544, () => {
  console.log('LLM dashboard running on http://localhost:5544');
});

Option B: CLI

If installed globally (or via npx), you can run the dashboard directly:

# Port defaults to 5544, DB dir defaults to .db
llm-usage-ts dashboard --db .db --port 5544

# or with npx, depending on your npm setup/name:
npx llm-usage-ts dashboard --db .db --port 5544

Note: The exact CLI name depends on the package bin mapping.

API

The SDK mirrors the Vercel AI SDK surface and adds an optional meta parameter to record tags and context.

Class: LLM

new LLM(options)

• options.dbDir: string (required) — directory where the SQLite file will be created
• options.redact: (s: string) => string (optional) — transform/redact prompts/messages before persisting

Methods

• generateText(options, meta?)
• streamText(options, meta?)
• generateObject(options, meta?)
• streamObject(options, meta?)

Where options are the same as in the Vercel AI SDK and include the provider model, e.g. model: openai('gpt-4o-2024-08-06'). The optional meta supports:

type LLMMeta = {
  tags?: string | string[]; // stored as JSON array for filtering
  contextId?: string;       // available for your own usage; not used by the SDK
  extra?: Record<string, unknown>;
};

Additional methods:

• updatePrices(next: PriceTable): merge/override the current price table.

Types

type Usage = {
  inputTokens?: number;
  outputTokens?: number;
  cachedInputTokens?: number;
  reasoningTokens?: number;
  totalTokens?: number;
};

type Price = {
  input: number;      // price per token
  output: number;     // price per token
  cacheInput?: number;// price per cached input token (defaults to input if omitted)
  reasoning?: number; // price per reasoning token (defaults to output if omitted)
};

type PriceTable = Record<string, Price>;

Server utilities

• new UsageDB(dir: string) — creates/opens the DB and runs migrations
• makeServer(db: UsageDB) — returns an Express app that renders the dashboard

Pricing

The SDK includes a default PriceTable for common OpenAI and Gemini models (see src/constants.ts). Prices are token-based, not per million tokens.

You can override or augment at runtime:

import { LLM } from 'llm-usage-ts/dist/llm.js';

const llm = new LLM({ dbDir: './.db' });
llm.updatePrices({
  'my-new-model': { input: 0.000001, output: 0.000002, cacheInput: 0.00000025, reasoning: 0.000002 },
});

Notes:

• For supported OpenAI models, the SDK excludes reasoning tokens from output token cost to avoid double-charging.
• If a model is unknown, cost will compute to 0 until you provide prices.

Data model

SQLite tables created in <dbDir>/llm-usage.sqlite:

• llm_calls — one row per call (kind, provider, model, tags, prompt/messages, output, status, error, latency, timestamps, provider metadata)
• llm_usage — token usage breakdown per call
• llm_cost — computed costs per call (input/cache/output/reasoning/total)

Tags are stored as JSON and can be filtered in the dashboard. Prompts or messages can be redacted using the redact option.

Redaction

Provide a function to scrub sensitive content before it is written:

const llm = new LLM({
  dbDir: './.db',
  redact: (s) => s
    .replace(/[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+/g, '[REDACTED_EMAIL]')
    .replace(/sk-[A-Za-z0-9_-]{20,}/g, '[REDACTED_KEY]'),
});

CLI reference (dashboard)

llm-usage-ts dashboard [options]

Options:
  -d, --db <dir>     DB directory (default: .db)
  -p, --port <port>  Port (default: 5544)

If you prefer, you can embed the server in your app using the programmatic API shown above.

Provider setup

You bring your own provider clients via the Vercel AI SDK adapters. Examples:

• OpenAI: @ai-sdk/openai → createOpenAI({ apiKey: process.env.OPENAI_API_KEY })
• Gemini: @ai-sdk/google → createGoogleGenerativeAI({ apiKey: process.env.GOOGLE_API_KEY })

Environment variables are handled by your app; this SDK does not read them directly.

Security note

Do not hardcode provider API keys in source code. Use environment variables and secret management. If prompts/messages may contain sensitive data, provide a redact function in LLM to scrub before persistence.

Caveats and notes

• This library wraps the AI SDK functions and records results; it does not alter model behavior.
• Provider metadata (if available) is stored as JSON for inspection.
• Streaming methods record usage and cost on completion via the SDK stream onFinish hook.

Acknowledgements

Built on top of the excellent Vercel AI SDK (ai).