Why Tollgate?

AI products bill customers on plans (per ticket, per seat, usage-based) but pay providers per token. Tollgate joins the two in real time — giving you per-customer, per-agent, per-run gross margin the moment each LLM call completes.

• 2-line integration — wrap your provider client once; every call is tracked automatically.
• Zero runtime dependencies — ships as a single ESM + CJS bundle.
• Non-blocking — usage reporting is fire-and-forget with automatic retries. Failures never break your LLM calls.
• Privacy-first — no prompt content is ever transmitted. Only token counts, model identifiers, and metadata.
• Universal coverage — Anthropic, OpenAI, Google Gemini, AWS Bedrock, and every OpenAI-compatible gateway.

┌──────────────┐    ┌───────────────┐    ┌────────────────┐
│  Your App    │───>│ LLM Provider  │───>│   Provider     │
│  (SDK wrap)  │<───│ (Anthropic,   │<───│   Response     │
│              │    │  OpenAI, …)   │    │  (tokens, id)  │
└──────┬───────┘    └───────────────┘    └────────────────┘
       │
       │  POST /api/track (background, non-blocking)
       v
┌─────────────────────────────────────────────────────┐
│  Tollgate Server                                    │
│                                                     │
│  ┌─────────────┐ ┌───────────┐ ┌─────────────────┐  │
│  │ Rate Card   │ │ Plan      │ │ Margin Rollups  │  │
│  │ (1,500+     │ │ Revenue   │ │ (per customer,  │  │
│  │  models)    │ │ Config    │ │  agent, run)    │  │
│  └─────────────┘ └───────────┘ └─────────────────┘  │
└─────────────────────────────────────────────────────┘

Installation

npm install @tollgateai/sdk

pnpm add @tollgateai/sdk   # or yarn add @tollgateai/sdk

Requirements: Node.js 18+ · Zero runtime dependencies · ESM and CommonJS supported

Quick Start

import Anthropic from '@anthropic-ai/sdk';
import { createTollgateClient, wrapAnthropic } from '@tollgateai/sdk';

const tollgate = createTollgateClient();          // reads TOLLGATE_API_KEY from env
const anthropic = wrapAnthropic(new Anthropic(), tollgate, {
  customerId: 'cust_acme',
  runId: 'ticket_8842',
});

// Every call is tracked automatically — tokens, cost, latency, tool calls.
const msg = await anthropic.messages.create({
  model: 'claude-sonnet-4-6',
  max_tokens: 1024,
  messages: [{ role: 'user', content: 'Resolve this billing dispute...' }],
});

// Close the run and book revenue.
await tollgate.resolve({
  runId: 'ticket_8842',
  customerId: 'cust_acme',
  outcome: 'resolved',
  revenueUnitCents: 50,       // $0.50 per resolved ticket
});

Provider Support

| Provider | Wrapper | Streaming | Extracted Fields | |---|---|---|---| | Anthropic | wrapAnthropic | Automatic | Input/output tokens, cache read/write, web search requests, tool calls, latency | | OpenAI | wrapOpenAI | stream_options: { include_usage: true } | Input/output tokens, reasoning, cached, audio in/out, text in/out, prediction tokens, service tier, tool calls, latency | | Google Gemini | wrapGemini | Automatic | Input/output tokens, thinking, cached, audio/image/video per-modality, web search (grounding), tool calls, latency | | OpenAI-compatible | wrapOpenAI + provider: 'openai_compatible' | Same as OpenAI | Same as OpenAI + gateway-reported cost (when available) | | AWS Bedrock | wrapBedrock | Automatic | Input/output tokens, cache read/write (per-TTL split), tool calls, latency |

Configuration

Environment Variables

| Variable | Required | Default | Description | |---|---|---|---| | TOLLGATE_API_KEY | Yes | — | Your account API key (tg_live_…) | | TOLLGATE_BASE_URL | No | https://www.tollgateai.dev | Self-hosted deployment URL |

Programmatic Configuration

const tollgate = createTollgateClient({
  apiKey: 'tg_live_xxx',
  baseUrl: 'https://www.tollgateai.dev',
  timeoutMs: 10_000,   // per-request timeout (default 10s)
  maxRetries: 2,        // retries on 5xx/429/network (default 2)
});

Auto-Instrumentation

Wrap your provider client once. Every create / generateContent / send call reports usage in the background — non-blocking, fire-and-forget. Failures go to onError (default: console.warn) and never break your LLM call.

Anthropic

import Anthropic from '@anthropic-ai/sdk';
import { createTollgateClient, wrapAnthropic } from '@tollgateai/sdk';

const tollgate = createTollgateClient();
const anthropic = wrapAnthropic(new Anthropic(), tollgate, {
  customerId: 'cust_acme',
  runId: 'ticket_8842',
});

await anthropic.messages.create({
  model: 'claude-sonnet-4-6',
  max_tokens: 512,
  messages: [{ role: 'user', content: 'Summarize this ticket...' }],
});

OpenAI

import OpenAI from 'openai';
import { createTollgateClient, wrapOpenAI } from '@tollgateai/sdk';

const tollgate = createTollgateClient();
const openai = wrapOpenAI(new OpenAI(), tollgate, { customerId: 'cust_acme' });

await openai.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'Hello' }],
});

Google Gemini

import { GoogleGenerativeAI } from '@google/generative-ai';
import { createTollgateClient, wrapGemini } from '@tollgateai/sdk';

const tollgate = createTollgateClient();
const genai = new GoogleGenerativeAI(process.env.GEMINI_API_KEY!);
const model = wrapGemini(
  genai.getGenerativeModel({ model: 'gemini-2.0-flash' }),
  tollgate,
  { customerId: 'cust_acme' },
);

const result = await model.generateContent('Explain quantum computing');

OpenAI-Compatible Gateways

Works with any OpenAI-compatible endpoint — OpenRouter, Groq, Together, Nebius, Vercel AI Gateway, local vLLM, and more.

import OpenAI from 'openai';
import { createTollgateClient, wrapOpenAI } from '@tollgateai/sdk';

const tollgate = createTollgateClient();
const groq = wrapOpenAI(
  new OpenAI({ apiKey: process.env.GROQ_API_KEY, baseURL: 'https://api.groq.com/openai/v1' }),
  tollgate,
  { customerId: 'cust_acme', provider: 'openai_compatible' },
);

await groq.chat.completions.create({
  model: 'llama-3.3-70b-versatile',
  messages: [{ role: 'user', content: 'Hello' }],
});

When a gateway returns cost inline (e.g. OpenRouter's usage.cost), the SDK captures it automatically as providerCostCents. The server uses it verbatim, bypassing the rate card. Gateways that don't return cost fall through to rate-card pricing. An explicit providerCostCents in the wrapper options always takes precedence.

AWS Bedrock

import { BedrockRuntimeClient, ConverseCommand } from '@aws-sdk/client-bedrock-runtime';
import { createTollgateClient, wrapBedrock } from '@tollgateai/sdk';

const tollgate = createTollgateClient();
const bedrock = wrapBedrock(
  new BedrockRuntimeClient({ region: 'us-east-1' }),
  tollgate,
  { customerId: 'cust_acme' },
);

await bedrock.send(new ConverseCommand({
  modelId: 'anthropic.claude-3-5-sonnet-20241022-v2:0',
  messages: [{ role: 'user', content: [{ text: 'Hello' }] }],
}));

Streaming

Streaming is captured automatically. Iterate the stream as usual — usage and latency are reported when the stream ends.

OpenAI / compatible requires stream_options: { include_usage: true }. Anthropic, Gemini, and Bedrock need no extra flags.

const stream = await openai.chat.completions.create({
  model: 'gpt-4o',
  stream: true,
  stream_options: { include_usage: true },
  messages: [{ role: 'user', content: 'Hello' }],
});
for await (const chunk of stream) { /* render to UI */ }
// Usage + latency reported automatically when stream ends.

Tracked Fields

Every auto-instrumented call captures these fields from the provider response:

| Field | Providers | Description | |---|---|---| | tokensIn | All | Input tokens (deduplicated — excludes cached/audio for OpenAI; excludes cached/audio/image/video for Gemini) | | tokensOut | All | Output tokens (deduplicated — excludes reasoning/audio for OpenAI; excludes audio/image for Gemini) | | reasoningTokens | OpenAI, Gemini | Reasoning/thinking tokens (billed at reasoning rate) | | cachedTokens | All | Prompt cache read tokens (reduced rate) | | cacheWrite5mTokens | Anthropic, Bedrock | Cache creation tokens (5-minute TTL) | | cacheWrite1hTokens | Bedrock | Cache creation tokens (1-hour TTL) | | audioTokensIn / Out | OpenAI, Gemini | Audio modality tokens (GPT-4o audio, Gemini multimodal) | | imageTokensIn / Out | Gemini | Image/vision input and generation output tokens | | videoTokensIn | Gemini | Video input tokens | | textTokensIn / Out | OpenAI, Gemini | Text-only modality tokens | | webSearchRequests | Anthropic, Gemini | Web search requests (server tools / grounding) | | acceptedPredictionTokens | OpenAI | Predicted Outputs: accepted tokens | | rejectedPredictionTokens | OpenAI | Predicted Outputs: rejected (waste) tokens | | serviceTier | OpenAI | Service tier (default, flex, priority) | | latencyMs | All | SDK-measured request duration in milliseconds | | toolCalls | All | Number of tool calls in the response | | providerCostCents | OpenAI-compatible | Gateway-reported cost — used verbatim, bypasses rate card | | model | All | Model identifier as reported by the provider |

Cost is computed server-side from token counts and a rate card that auto-syncs daily from the LiteLLM registry (1,500+ models). Rate cards include per-token pricing for every modality, cache tier, reasoning, and web search. Unknown models are priced at $0 and flagged in logs.

Provider Field Coverage

| Anthropic API Field | SDK Field | Notes | |---|---|---| | usage.input_tokens | tokensIn | Input tokens (excludes cached) | | usage.output_tokens | tokensOut | Output tokens (includes reasoning — billed at output rate) | | usage.cache_read_input_tokens | cachedTokens | Prompt cache read tokens | | usage.cache_creation_input_tokens | cacheWrite5mTokens | Prompt cache creation tokens | | usage.server_tool_use.web_search_requests | webSearchRequests | Web search server tool requests | | response.content[] (type tool_use) | toolCalls | Count of tool-use content blocks | | (SDK-measured) | latencyMs | Request duration |

Anthropic bills reasoning tokens at the output rate. The SDK reports the full output_tokens count; the server-side rate card applies the matching output rate.

In streaming mode, message_start carries input/cache counts and message_delta carries the output count. The SDK accumulates both automatically.

| OpenAI API Field | SDK Field | Notes | |---|---|---| | usage.prompt_tokens | tokensIn | Minus cached and audio tokens to prevent double-billing | | usage.completion_tokens | tokensOut | Minus reasoning and audio tokens to prevent double-billing | | usage.completion_tokens_details.reasoning_tokens | reasoningTokens | Reasoning/thinking tokens | | usage.prompt_tokens_details.cached_tokens | cachedTokens | Prompt cache read tokens | | usage.prompt_tokens_details.audio_tokens | audioTokensIn | Audio input tokens | | usage.completion_tokens_details.audio_tokens | audioTokensOut | Audio output tokens | | usage.prompt_tokens_details.text_tokens | textTokensIn | Text modality input tokens | | usage.completion_tokens_details.text_tokens | textTokensOut | Text modality output tokens | | usage.completion_tokens_details.accepted_prediction_tokens | acceptedPredictionTokens | Predicted Outputs: accepted | | usage.completion_tokens_details.rejected_prediction_tokens | rejectedPredictionTokens | Predicted Outputs: rejected | | service_tier | serviceTier | Service tier used | | choices[].message.tool_calls | toolCalls | Tool call count | | (SDK-measured) | latencyMs | Request duration |

OpenAI's prompt_tokens and completion_tokens are totals that include sub-category tokens. The SDK subtracts each sub-category so every token is costed at exactly one rate.

| Google API Field | SDK Field | Notes | |---|---|---| | usageMetadata.promptTokenCount | tokensIn | Minus cached, audio, image, video to prevent double-billing | | usageMetadata.candidatesTokenCount | tokensOut | Minus audio and image output (thinking is already excluded by Google) | | usageMetadata.thoughtsTokenCount | reasoningTokens | Thinking/reasoning tokens (Gemini 2.x) | | usageMetadata.cachedContentTokenCount | cachedTokens | Prompt cache read tokens | | promptTokensDetails[AUDIO] | audioTokensIn | Audio input modality | | candidatesTokensDetails[AUDIO] | audioTokensOut | Audio output modality | | promptTokensDetails[IMAGE] | imageTokensIn | Image/vision input | | candidatesTokensDetails[IMAGE] | imageTokensOut | Image generation output | | promptTokensDetails[VIDEO] | videoTokensIn | Video input | | promptTokensDetails[TEXT] | textTokensIn | Text input | | candidatesTokensDetails[TEXT] | textTokensOut | Text output | | candidates[].groundingMetadata.webSearchQueries | webSearchRequests | Google Search grounding | | candidates[].content.parts[].functionCall | toolCalls | Function call count | | (SDK-measured) | latencyMs | Request duration |

Google's candidatesTokenCount does not include thoughtsTokenCount, so reasoning tokens are not subtracted. However, it does include audio and image output tokens, so the SDK subtracts those to prevent double-billing.

| Bedrock API Field | SDK Field | Notes | |---|---|---| | usage.inputTokens | tokensIn | Input tokens | | usage.outputTokens | tokensOut | Output tokens (includes reasoning — Bedrock does not split) | | usage.cacheReadInputTokens | cachedTokens | Prompt cache read tokens | | usage.cacheDetails[ttl="5m"] | cacheWrite5mTokens | Cache creation (5-minute TTL) | | usage.cacheDetails[ttl="1h"] | cacheWrite1hTokens | Cache creation (1-hour TTL, higher rate) | | output.message.content[].toolUse | toolCalls | Tool-use content block count | | (SDK-measured) | latencyMs | Request duration |

Bedrock's cacheDetails array provides per-TTL breakdowns. The SDK splits these into cacheWrite5mTokens and cacheWrite1hTokens. When cacheDetails is absent, cacheWriteInputTokens falls back to the 5m bucket.

In streaming mode (ConverseStream), the final metadata event carries usage totals. Tool calls are accumulated from contentBlockStart events.

Pricing Models

Per-Call Revenue

For simple per-call billing, pass revenueUnitCents in the wrapper options:

const anthropic = wrapAnthropic(new Anthropic(), tollgate, {
  customerId: 'cust_acme',
  revenueUnitCents: 50,  // $0.50 earned per LLM call
});

Outcome-Based Pricing

Under per-resolution pricing, only a resolved run earns revenue. Escalated or failed runs earn $0, but provider costs still count against margin.

const runId = 'ticket_8842';
const anthropic = wrapAnthropic(new Anthropic(), tollgate, {
  customerId: 'cust_acme',
  runId,
});

// ... multiple LLM calls within this run ...

await tollgate.resolve({
  runId,
  customerId: 'cust_acme',
  outcome: 'resolved',        // 'resolved' | 'escalated' | 'failed'
  revenueUnitCents: 50,
});

External Tool Costs

Report costs from non-LLM services (image generation, code sandboxes, search APIs) alongside LLM calls:

await tollgate.track({
  customerId: 'cust_acme',
  runId: 'ticket_8842',
  provider: 'openai',
  model: 'dall-e-3',
  tokensIn: 0,
  tokensOut: 0,
  externalCostCents: 4.0,     // $0.04 for the DALL-E call
  idempotencyKey: 'ticket_8842#dalle',
});

Customer & Plan Setup

Create customers and assign plans before sending usage so plan-priced revenue is recognized from the first event. Idempotent — safe to call on every app boot.

await tollgate.upsertCustomer({
  customerId: 'cust_acme',
  name: 'Acme Corp',
  plan: {
    name: 'Pro Plan',
    pricingModel: 'usage_based',   // per_unit | per_resolution | usage_based | per_seat | flat | hybrid
    unitRevenueCents: 10,
  },
});

Error Handling

The SDK separates tracking errors (non-fatal) from client errors (actionable):

// Tracking errors are swallowed by default (console.warn).
// Override with onError to route to your observability stack:
const anthropic = wrapAnthropic(new Anthropic(), tollgate, {
  customerId: 'cust_acme',
  onError: (err) => Sentry.captureException(err),
});

// Client errors (missing API key, invalid plan) throw TollgateError:
import { TollgateError } from '@tollgateai/sdk';

try {
  await tollgate.upsertCustomer({ customerId: 'cust_acme' });
} catch (err) {
  if (err instanceof TollgateError) {
    console.error(err.status, err.body);  // HTTP status + response body
  }
}

Retry behavior: The client retries on 5xx and 429 responses with exponential backoff (200ms, 400ms, ...). Deterministic 4xx errors (400, 401, 403, 404, 422) fail immediately.

API Reference

Exports

// Client
createTollgateClient(options?)   // -> TollgateClient
TollgateError                    // Error with status & body

// Auto-instrumentation wrappers
wrapAnthropic(client, tollgate, options)       // -> instrumented Anthropic client
wrapOpenAI(client, tollgate, options)          // -> instrumented OpenAI / compatible client
wrapBedrock(client, tollgate, options)         // -> instrumented Bedrock Runtime client
wrapGemini(model, tollgate, options)           // -> instrumented Gemini model

// Low-level event builders (for manual track payloads)
anthropicEventFrom(msg, options)               // -> TrackEventInput | null
openAIEventFrom(completion, options)           // -> TrackEventInput | null
bedrockEventFrom(usage, model, options)        // -> TrackEventInput | null
geminiEventFrom(response, options)             // -> TrackEventInput | null

// Types
Provider         // 'anthropic' | 'openai' | 'openai_compatible' | 'bedrock' | 'google'
RunOutcome       // 'resolved' | 'escalated' | 'failed'
PricingModel     // 'per_unit' | 'per_resolution' | 'usage_based' | 'per_seat' | 'flat' | 'hybrid'
TrackEventInput  // Full event payload type

TollgateClient

| Method | Description | |---|---| | track(event: TrackEventInput) | Report a single usage event. Idempotent on idempotencyKey. Returns { status, eventId }. | | resolve(input: ResolveInput) | Close a run with an outcome. Books revenue only when outcome === 'resolved'. | | upsertCustomer(input: UpsertCustomerInput) | Create or update a customer and optionally assign a plan. Returns { status, customerId, id, planId }. |

TollgateClientOptions

| Field | Type | Default | Description | |---|---|---|---| | apiKey | string | TOLLGATE_API_KEY env | Account API key | | baseUrl | string | https://www.tollgateai.dev | Tollgate server URL | | timeoutMs | number | 10000 | Per-request timeout in milliseconds | | maxRetries | number | 2 | Retry attempts on 5xx / 429 / network errors | | fetch | typeof fetch | globalThis.fetch | Custom fetch implementation |

InstrumentOptions

| Field | Type | Required | Description | |---|---|---|---| | customerId | string | Yes | Your end customer's stable identifier | | agentId | string | No | Agent or workflow identifier | | runId | string \| () => string | No | Logical run ID (defaults to provider response ID) | | provider | Provider | No | Override the reported provider | | revenueUnitCents | number \| (response) => number | No | Revenue per call in cents | | providerCostCents | number \| (response) => number | No | Exact cost override in cents (skips rate card) | | onError | (err) => void | No | Error handler for background tracking (default: console.warn) |

How It Works

1. Proxy wrappers intercept provider calls without modifying the request or response. Your code sees the exact same types and behavior as without the SDK.
2. After the provider responds, the wrapper extracts token counts by modality, tool calls, service tier, and latency from the response object.
3. A POST /api/track fires in the background with automatic retries on transient failures. Your application code continues immediately.
4. The server computes cost from tokens via rate cards (per modality, cache tier, reasoning, and web search), joins it with plan-configured revenue, and updates real-time margin rollups.
5. Events are idempotent — deduplication is based on idempotencyKey (auto-set to the provider response ID).

Security & Privacy

• No prompt content is ever transmitted. Only token counts, model identifiers, and metadata.
• Idempotent ingestion — duplicate events are safely deduplicated server-side.
• Non-invasive — background tracking never throws into your application code.
• Transport security — all communication over HTTPS with Bearer token authentication.

License

MIT — see LICENSE for details.