@browserstack/ai-sdk

A TypeScript/NodeJS SDK for AI application observability, testing, and evaluation.

Requires Node.js >= 18.

Installation

npm install @browserstack/ai-sdk@beta

Quick Start

import { AISDK } from '@browserstack/ai-sdk';

// Initialize the client
const client = new AISDK({
  publicKey: 'your-public-key',
  secretKey: 'your-secret-key',
});

// Create a trace
const trace = client.trace({
  name: 'user_login_flow',
  metadata: {
    testSuite: 'authentication_tests',
    testFile: 'auth.test.js',
  },
  tags: ['auth', 'critical'],
});

// Create spans
const loginSpan = trace.span({
  name: 'navigate_to_login',
  input: { url: '/login' }
});

loginSpan.end({
  output: { pageLoaded: true, loadTime: 250 }
});

// Create events
trace.event({
  name: 'assertion_check',
  metadata: {
    assertion: 'Login button should be visible',
    passed: true,
    expected: 'button visible',
    actual: 'button visible'
  }
});

// Cleanup
await client.flushAsync();
await client.shutdownAsync();

Auto-Instrumentation

Automatically trace LLM calls without code changes:

node --require @browserstack/ai-sdk/instrument app.js

Configuration

The client can be configured using environment variables or constructor options:

Environment Variables

AISDK_PUBLIC_KEY=your_public_key
AISDK_SECRET_KEY=your_secret_key

Memory and backpressure

The SDK holds trace data in memory between your call and a successful delivery. When ingestion is slow or failing, that held data is bounded by a byte budget — once the budget is reached the SDK sheds the oldest traces rather than growing until the process is killed.

| Variable | Default | What it controls | | --- | --- | --- | | AISDK_MAX_HELD_BYTES | 104857600 (100 MB) | Total estimated bytes held across the queue and in-flight requests. The one knob to change if you need a different memory ceiling. | | AISDK_MAX_QUEUE_SIZE | 1000 | Items held before delivery. An independent cap — whichever binds first wins. | | AISDK_MAX_SOCKETS | 30 | Connection pool size. | | AISDK_BATCH_SIZE | 15 | Items per request. | | AISDK_BATCH_TIMEOUT_MS | 5000 | Longest a partial batch waits before being sent. | | AISDK_MAX_RETRIES | 3 | Retries after a failed attempt, so 4 total attempts. | | AISDK_REQUEST_TIMEOUT_MS | 30000 | Per-attempt request timeout. 0 disables it entirely. |

Sizing your container. AISDK_MAX_HELD_BYTES bounds the SDK's tracked payload bytes, not the process. Expect real RSS of roughly 1.8x the budget — a sustained-failure load test against the 100 MB default settles flat at ~180 MB. Budget for at least 2x on top of your application's own footprint.

Half the budget is reserved for in-flight requests and half for the queue, so a brief slowdown is absorbed rather than shed: queued items can still be delivered when the endpoint recovers, while in-flight ones are pinned until their retries finish.

Set plain byte counts. Suffixes are not parsed — AISDK_MAX_HELD_BYTES=100MB reads as 100 bytes, not 100 megabytes, and 1e6 reads as 1. Values below 64 KB are refused with a [CONFIG_REJECTED] line and the default is used instead.

Span attribute limits

OpenTelemetry defaults to 128 attributes per span, which a request advertising many tools (or a long agentic loop) can approach. The SDK raises this ceiling to 10000 by default — a backstop against an unbounded producer, not a payload guard.

| Variable | What it controls | | --- | --- | | OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT | Set to a number to use your own span attribute count limit instead of the SDK's default. | | OTEL_ATTRIBUTE_COUNT_LIMIT | Same override; OpenTelemetry's general (non-span-specific) variable, read when the one above is unset. |

Reading the logs

Four lines are printed by default, without AISDK_DEBUG, because they mean trace data is being discarded. Each is rate-limited to one line per minute and carries suppressed=N for the occurrences it stands for.

| Line | Meaning | What to do | | --- | --- | --- | | [BUDGET_SHED] | The byte budget was reached; oldest traces dropped. | Ingestion is not keeping up. Check endpoint health first — raising the budget only delays this. | | [QUEUE_OVERFLOW] | The item cap was reached. | Raise AISDK_MAX_QUEUE_SIZE, or check endpoint health. | | [INFLIGHT_CAP] | Dispatch paused because in-flight bytes are at their ceiling. | Normal backpressure. Sustained appearances mean ingestion is degraded. | | [DROP] | A batch was discarded after exhausting its retries. | Delivery was never confirmed. The data may still have reached the server — X-Batch-Id lets ingestion dedupe a retried batch — so treat this as unconfirmed, not as certain loss. |

Everything else, including per-request diagnostics, requires AISDK_DEBUG=true.

Constructor Options

const client = new AISDK({
  publicKey: 'your_public_key',
  secretKey: 'your_secret_key',
  environment: 'production',
  flushAt: 20,
  flushInterval: 10000,
});