@heystack/otel

Runtime-aware OpenTelemetry tracing that exports to Heystack. One import per runtime, so the . entry stays safe to load anywhere (including Edge/Workers) without dragging in the Node SDK.

npm install @heystack/otel

Always read your key from the environment — never paste it into source:

# .env
HEYSTACK_API_KEY=sk_live_…

Requires @heystack/otel >=0.3.0 (prefer >=0.3.2). 0.3.2 fixes the no-op suppressTracing (feedback-loop guard) on the Workers/Next-on-OpenNext path and adds the Workers nodejs_compat requirement. See Migration below.

Runtime matrix

| Runtime | Import | Notes | | --- | --- | --- | | Next.js — any deploy target (Vercel/Node and Cloudflare/OpenNext) | @heystack/otel/next | registerHeystack in instrumentation.ts. Auto-detects Node vs Cloudflare workerd and picks the right exporter. No-op on Edge. | | Standalone Cloudflare Workers (hand-written export default { fetch }) | @heystack/otel/workers | instrument() wraps your handler. Fetch-based exporter, flushes via ctx.waitUntil. | | Node / Express / Fastify / NestJS (long-running server) | @heystack/otel/node | initHeystack: auto-instrumentations + graceful shutdown. | | Browser (SPA / any web frontend) | @heystack/otel/web | instrumentWeb: session replay + opt-in browser distributed tracing (tracing: true) that emits CLIENT spans + propagates W3C traceparent (browser→API shows as one trace). No-op on the server (SSR-safe). | | Anywhere (pure helpers) | @heystack/otel | buildExporterConfig, types. No Node SDK loaded. |

Release & commit attribution (version / build)

Two optional options tag every span with which release and commit it came from, so the console can power release health, suspect release, and suspect commit views (spot the exact version/commit that introduced a regression). They work on every runtime entry (/node, /next, /workers).

initHeystack({
  apiKey: process.env.HEYSTACK_API_KEY,
  service: "my-app",
  version: process.env.APP_VERSION,          // → service.version (e.g. "1.4.2" or a git tag)
  build: process.env.GIT_SHA,                // → service.build   (the commit SHA)
});

| Option | Resource attribute | Meaning | | --- | --- | --- | | version | service.version | Release identifier — a semver (1.4.2), a git tag, or any string that changes per release. Groups telemetry by release for release health + suspect release. | | build | service.build | The commit SHA of this deploy. Powers suspect commit — when you configure a repository URL for the app in the console, the SHA deep-links to the exact commit. |

Both are optional and only emitted when set — an empty value is skipped (the console's release queries filter these out). Set them from your build/deploy environment, e.g. build: process.env.GIT_SHA (GIT_SHA=$(git rev-parse HEAD) in CI).

On /node they also fall back to environment variables when the option is omitted: HEYSTACK_SERVICE_VERSION (or OTEL_SERVICE_VERSION) for the release and HEYSTACK_SERVICE_BUILD for the commit — so a Node deploy can attribute a release without touching code:

HEYSTACK_SERVICE_VERSION=1.4.2
HEYSTACK_SERVICE_BUILD=$(git rev-parse HEAD)

On /next pass them to registerHeystack({ service, version, build }) — they're threaded to whichever runtime the app runs on (Node or workerd). On /workers pass them in the instrument() / initHeystackWorkers() config.

Node / Express / etc.

At the very top of your app's entry file:

import { initHeystack } from "@heystack/otel/node";

initHeystack({
  apiKey: process.env.HEYSTACK_API_KEY,
  service: "my-app",
  version: process.env.APP_VERSION, // optional: release health + suspect release/commit
  build: process.env.GIT_SHA,       // optional: commit SHA (see "Release & commit attribution")
});

This enables auto-instrumentations (HTTP, Express, etc.) so you get spans without manual wiring. version/build are optional (see Release & commit attribution).

Slimming down auto-instrumentations (cost)

The default getNodeAutoInstrumentations() eagerly patches ~40 libraries (HTTP, DNS, fs, net, gRPC, and every popular DB/HTTP client). That adds startup time and per-call overhead even for libraries your app never uses. To load only what you need, pass your own instrumentations array:

import { initHeystack } from "@heystack/otel/node";
import { HttpInstrumentation } from "@opentelemetry/instrumentation-http";
import { ExpressInstrumentation } from "@opentelemetry/instrumentation-express";

initHeystack({
  apiKey: process.env.HEYSTACK_API_KEY,
  service: "my-app",
  instrumentations: [new HttpInstrumentation(), new ExpressInstrumentation()],
});

instrumentations defaults to [getNodeAutoInstrumentations()]. Passing your own array replaces the default entirely (it's ignored when autoInstrument: false). initHeystack is also idempotent as of 0.3.2 — calling it again returns the already-started SDK (no duplicate instrumentations or signal handlers, which matters across Next dev-server reloads).

Next.js (any deploy target, including Cloudflare/OpenNext)

In instrumentation.ts at the project root:

export async function register() {
  const { registerHeystack } = await import("@heystack/otel/next");
  await registerHeystack({ service: "my-app" });
}

apiKey defaults to process.env.HEYSTACK_API_KEY. registerHeystack is runtime-aware and safe to call unconditionally:

• Vercel / Node deploys → uses the Node OTel SDK (auto-instrumentations).
• Cloudflare / OpenNext (workerd) → Next still reports NEXT_RUNTIME === "nodejs", but the code actually runs on workerd where the Node SDK's node:http OTLP exporter can't send (it initializes and silently exports nothing). registerHeystack detects workerd and registers the fetch-based exporter instead, so framework spans actually ship. No separate setup or import needed.
• Edge runtime → no-op (neither SDK can run there).

Standalone Cloudflare Workers

For a hand-written Worker (export default { fetch }), wrap the handler with instrument():

import { instrument } from "@heystack/otel/workers";

export default instrument(
  {
    async fetch(req, env, ctx) {
      return new Response("ok");
    },
  },
  {
    service: "my-worker",      // apiKey defaults to env.HEYSTACK_API_KEY
    getUser: (req) => ({
      id: req.headers.get("x-user-id") ?? undefined,
    }),
    instrumentBindings: true,  // auto-trace D1/KV/R2/Vectorize/AI/Queues/Service bindings
  },
);

instrument() must be the outermost wrapper if other middleware also wraps the handler, so the request span covers everything inside:

export default instrument(withOtherMiddleware(worker), { service: "my-worker" });

Set the key as a secret: wrangler secret put HEYSTACK_API_KEY.

Requires nodejs_compat on workerd. The SDK uses AsyncLocalStorage for per-request context isolation. Add to wrangler.toml:

compatibility_flags = ["nodejs_compat"]

Without it the SDK falls back to a synchronous stack-based context manager — suppression still works, and binding spans (D1/KV/R2/Vectorize/AI/Queue/Service) are still parented to the request's SERVER span (they fall back to the request context explicitly, so they never orphan into their own single-span traces). What still degrades to best-effort without nodejs_compat is cross-await parenting for outbound-fetch CLIENT spans and manual withSpan spans — so nodejs_compat remains recommended for a fully-nested trace tree.

WorkersConfig options

| Option | Type | Notes | | --- | --- | --- | | service | string | Required. Service name that appears in the Heystack console. | | apiKey | string? | Defaults to env.HEYSTACK_API_KEY. | | getUser | (req: Request) => { id?, session?, requestId? } \| undefined | Called per request. id → enduser.id, session → session.id, requestId → http.request.id (falls back to the cf-ray header). | | instrumentBindings | boolean \| string[] | true = auto child spans for all detected D1 / KV / R2 / Vectorize / Workers AI / Queue producer / Service binding bindings; string[] = only the named bindings. Default false. | | sampling | { rate?: number } \| { remote: true } | Head-sampling configuration. { rate }: keep a deterministic fraction of fresh root traces (0–1; default 1 = keep all). { remote: true }: fetch the rate from the Heystack config endpoint instead — lets you change it centrally without redeploying. Cold isolates keep all traffic until the first config fetch resolves; fails open if the config can't be reached. Parent-respecting in both modes: a request arriving with a sampled traceparent is always recorded. See Head sampling below. | | ai | { captureContent?: boolean; redact?: (text: string) => string; maxContentChars?: number } | LLM/gen_ai capture for outbound calls to known providers. See AI / LLM observability below. | | waitUntil | (p: Promise<unknown>) => void | Override the isolate keep-alive hook; defaults to the auto-detected ctx.waitUntil. | | version | string? | Release identifier → service.version resource attribute. Powers release health + suspect release. See Release & commit attribution. | | build | string? | Commit SHA → service.build resource attribute. Powers suspect commit (deep-links to the commit when a repo URL is set in the console). | | endpoint | string? | Override the ingest endpoint (advanced). |

Head sampling

sampling: { rate } performs head sampling — the keep/drop decision is made at the start of each fresh root trace, before any export. Traces that are dropped never leave the worker (no egress, no ingest cost, no storage). Traces that are kept are exported in full.

export default instrument(worker, {
  service: "my-api",
  sampling: { rate: 0.2 }, // keep ~20% of traffic
});

The sampler is deterministic by trace ID: the same trace always makes the same decision, so parent→child consistency is maintained when this worker is in a call chain sampled at the same rate by another service.

Tradeoff: head sampling can't guarantee keeping error traces — the drop/keep decision is made before the response status is known. If you need every error captured, keep rate: 1 (the default). Heystack's ingest-side keep rules (error and slow retention) apply among traces that are exported; they can't rescue a trace that was dropped in the worker before export.

The rate governs fresh root traces only (no inbound traceparent, or traceparent with sampled=0). A request arriving with a sampled inbound traceparent (sampled=1) is always recorded — the parent's decision is respected, so a distributed trace is never split mid-flight by the child's sample rate.

Remote sampling

sampling: { remote: true } lets Heystack control the rate centrally — change it from the dashboard without redeploying:

export default instrument(worker, {
  service: "my-api",
  sampling: { remote: true },
});

On startup the worker fetches its configured rate from the Heystack config endpoint. Cold isolates keep all traffic until that first fetch resolves (fails open — nothing is dropped while loading). If the config endpoint can't be reached, the worker keeps everything. The same parent-respecting rule applies: a request with an inbound sampled traceparent is always recorded regardless of the fetched rate.

Automatic tracing

instrument() traces the following automatically, with no additional config:

• Incoming requests (fetch) — a SERVER span per request, carrying http.request.method, url.full, url.path, server.address, http.response.status_code, and enduser.id/client.address/geo attributes (see Client enrichment below). An inbound W3C traceparent header is continued so the client and server share one trace; a traceparent response header (+ Access-Control-Expose-Headers: traceparent) is set so downstream clients can read it.
• Outbound fetch — each outbound subrequest while a request span is active gets a CLIENT child span (http.request.method, url.full, server.address, http.response.status_code). A W3C traceparent header is injected into the subrequest so a downstream Heystack-instrumented service continues the same trace (distributed tracing across services). The exporter's own ingest POST is never traced.
• Queue consumers (queue) — a CONSUMER span per batch, with messaging.destination.name (queue name) and messaging.batch.message_count.
• Scheduled handlers (scheduled) — an INTERNAL span per invocation, with controller.cron.
• Binding calls (when instrumentBindings is set) — a child span for every D1 query (db.statement), KV read/write, R2 operation, Vectorize query, Workers AI inference (gen_ai.* attributes including model name and token usage), Queue .send/.sendBatch (PRODUCER spans with messaging.* attributes), and Service binding .fetch calls (CLIENT spans with traceparent injected so calls to other Workers stitch into the same distributed trace).

Client enrichment

These attributes are set automatically on every SERVER span from request metadata:

| Attribute | Source | | --- | --- | | enduser.id | getUser(req).id | | session.id | getUser(req).session | | http.request.id | getUser(req).requestId or cf-ray header | | client.address | CF-Connecting-IP header | | geo.country, geo.region, geo.city, geo.asn | Cloudflare req.cf object |

AI / LLM observability

When your Worker calls an LLM API, instrument() automatically attaches OpenTelemetry gen_ai semantic conventions to the outbound CLIENT span. No extra code is needed — detection is based on the target hostname.

Detected providers:

• OpenAI — api.openai.com
• Anthropic — api.anthropic.com
• Cloudflare AI Gateway — any host ending in gateway.ai.cloudflare.com
• Google — generativelanguage.googleapis.com

Attributes captured automatically (metadata, always on):

| Attribute | Description | | --- | --- | | gen_ai.system | Provider name (openai, anthropic, cloudflare, google) | | gen_ai.request.model | Model from the request body (body.model) | | gen_ai.request.max_tokens | Max tokens from the request (max_tokens or max_completion_tokens) | | gen_ai.request.temperature | Temperature from the request | | gen_ai.response.model | Model from the response body | | gen_ai.response.id | Response ID | | gen_ai.usage.input_tokens | Input/prompt tokens used | | gen_ai.usage.output_tokens | Output/completion tokens used | | gen_ai.response.finish_reason | Stop reason (stop, end_turn, etc.) |

Streaming responses (text/event-stream) skip response enrichment (the body is not consumed) but request-side attributes are still set.

Content capture (opt-in, strongly recommended for AI-app RCA):

Prompt and completion text are not captured by default — enable ai.captureContent: true to capture them. This is strongly recommended when debugging AI app issues (wrong answers, unexpected outputs, prompt regressions) since without the content you can see that an LLM call happened and how long it took but not what was said.

Use redact to scrub sensitive fields before they leave the Worker. Values are truncated to maxContentChars (default 8000) before storage.

export default instrument(worker, {
  service: "my-ai-worker",
  ai: {
    captureContent: true,           // capture prompts + completions (off by default)
    redact: (text) =>               // optional: scrub sensitive text
      text.replace(/sk-[a-zA-Z0-9]+/g, "[REDACTED]"),
    maxContentChars: 4000,          // optional: cap per-value length (default 8000)
  },
});

When captureContent is enabled, two additional attributes appear on the CLIENT span:

| Attribute | Description | | --- | --- | | gen_ai.prompt | The request messages array (JSON-serialised, truncated) | | gen_ai.completion | The first choice/content block from the response |

Body safety guarantees:

• Request body: only read when it is already a plain string (the common LLM SDK case). Non-string bodies (streams, FormData, etc.) are never touched.
• Response body: read only from response.clone(), so the original response is returned untouched and your handler can call .json() / .text() normally.
• Parse failures or unexpected shapes degrade silently to the plain CLIENT span — no errors are thrown.
• For a detected provider with a JSON response, the cloned body is parsed before the fetch() promise resolves to your handler (the span must close with usage attributes). This adds negligible latency to an already-slow LLM call.
• The response size guard (512 KB) only applies when the response carries a content-length header; a chunked JSON response without one is still read. The known providers bound this via max_tokens, so it is safe in practice.

Manual spans: withSpan / addEvent

Inside a traced handler, add finer-grained spans without touching the OpenTelemetry API directly:

import { instrument, withSpan, addEvent } from "@heystack/otel/workers";

// Inside a fetch handler:
const result = await withSpan("parse-payload", { "source": "body" }, async (span) => {
  addEvent("parsing-started");
  span.setAttribute("content-type", req.headers.get("content-type") ?? "");
  return JSON.parse(await req.text());
});

// Without attrs:
const data = await withSpan("call-llm", async () => {
  return callMyAI();
});

withSpan(name, attrs?, fn) — runs fn inside a new child span parented to the currently-active span. The span is started before fn and ended in finally; exceptions are recorded on the span and re-thrown. The fn receives the live Span as its argument.

addEvent(name, attrs?) — adds a named event to the currently-active span. No-op when no span is active.

Why a ContextManager

context.with(...) in OpenTelemetry is a no-op unless a ContextManager is registered with the global API. Without one, suppressTracing() — the primary defence against the self-trace feedback loop — silently does nothing in production. As of 0.3.2 the SDK registers a ContextManager exactly once at init. With AsyncLocalStorage (workerd under nodejs_compat, Node) you also get cross-await parent→child span linking and per-request context isolation — concurrent requests no longer share or clobber the active span.

As of 0.3.1 instrument() forwards every other handler your Worker exports — queue, scheduled, tail, etc. — untouched, so wrapping never drops a handler Cloudflare requires for deploy (it previously returned only { fetch }, which broke Queue/Cron Workers). On top of forwarding, queue and scheduled are themselves traced when present: each gets a root span via the global tracer (queue <queueName> as a CONSUMER span with batch attributes; scheduled <cron> as an INTERNAL span with the cron attribute), flushed via ctx.waitUntil just like fetch.

Streaming responses & trace correlation (/workers)

instrument() keeps the SERVER span open until the response body finishes streaming, so a streamed response's duration includes time-to-last-byte and the span carries a first_byte event (time-to-first-byte). It also continues an inbound W3C traceparent (so a client request and the server handler share one trace) and returns a traceparent response header — plus Access-Control-Expose-Headers: traceparent so browser clients can read it.

Durable Objects are NOT covered by instrument()

instrument() wraps the keys of the default-export handler object (fetch/queue/scheduled/… ). Durable Objects are separate named class exports, so spreading the handler object does not touch them — a DO's fetch/alarm methods run untraced even when your Worker's default export is wrapped.

To trace a Durable Object, instrument it manually with the global tracer (which instrument() / initHeystackWorkers() already registered) and flush per invocation:

import { trace, SpanKind, context } from "@opentelemetry/api";
import { flushHeystack } from "@heystack/otel/workers";

export class Counter {
  async fetch(req: Request): Promise<Response> {
    const tracer = trace.getTracer("heystack");
    const span = tracer.startSpan(`DO ${new URL(req.url).pathname}`, {
      kind: SpanKind.SERVER,
    });
    try {
      return await context.with(trace.setSpan(context.active(), span), async () => {
        // ...your DO logic...
        return new Response("ok");
      });
    } finally {
      span.end();
      this.state.waitUntil(flushHeystack()); // ensure the export POST completes
    }
  }
}

The default export still needs to be wrapped with instrument() (or initHeystackWorkers() called) so the global provider + ContextManager are registered before the DO runs.

@heystack/otel/web (browser / session replay)

For a browser frontend (any SPA / web app), instrumentWeb records session replay, captures uncaught errors + console.error as logs, and injects a W3C traceparent header on outgoing fetch calls, so replays/errors correlate with the backend traces they triggered. It is a no-op on the server (SSR-safe), so it's safe to call from code that also runs during server rendering.

The rrweb recorder ships inside this package — there is nothing else to install. Uploads go cross-origin to the Heystack ingest endpoint and work out of the box (no CORS configuration on your side).

import { instrumentWeb } from "@heystack/otel/web";

const stop = await instrumentWeb({
  // A BROWSER-exposed ingest key — it ships to the client, like an analytics
  // write key. Use a public env var (below), ideally a dedicated key you can
  // rotate independently of your server-side key. NOT your server secret.
  apiKey: import.meta.env.VITE_HEYSTACK_API_KEY, // Vite; Next.js: process.env.NEXT_PUBLIC_HEYSTACK_API_KEY
  service: "my-web-app",
});

// later, e.g. on unmount / sign-out:
stop();

instrumentWeb returns a stop() function that ends the recording session.

Where to call it. It must run in the browser. In a Vite/CRA SPA, call it once in your client entry (main.tsx). In Next.js (App Router), wrap it in a small "use client" component that calls it from a useEffect and mount that once in your root layout (server components can't call it). Recording is server-gated: nothing is captured until you enable replay for the app in the console, so it's safe to ship this before flipping the switch.

Options

| Option | Type | Notes | | --- | --- | --- | | apiKey | string | Required. A Heystack ingest key, exposed to the browser (public env var). Prefer a dedicated key you can rotate — not your server-side secret. | | service | string | Required. The OTel service name (matches the app's service in the console). | | userId | string? | Optional app-supplied identifier stamped on the session. | | endpoint | string? | Optional ingest endpoint override (defaults to the Heystack ingest endpoint). | | sampleRate | number? | Optional local override for the recording sample rate (0–1). By default sampling is controlled from the console. | | flushIntervalMs | number? | How often buffered events are flushed (default 5000ms). | | flushEveryEvents | number? | Max buffered events before an early flush (default 200). | | tracing | boolean? | Opt in to browser distributed tracing (default off). Emits a real CLIENT span per outbound fetch and propagates W3C trace context, so browser→backend calls show as one connected trace + a service-map edge. Independent of replay. | | traceSampleRate | number? | Head sample rate for browser tracing (0–1, default 1 when tracing is on). Lower it to cap span volume/cost on busy apps. | | tracePropagationTargets | (string \| RegExp)[]? | Cross-origin backends allowed to receive the traceparent header (same-origin always is). Trace context is never sent to any other origin, so third-party calls (Clerk, Stripe, analytics) aren't broken by an unexpected CORS preflight. Match by substring/RegExp against the URL, e.g. ["api.myapp.com"]. A listed backend must allow traceparent/tracestate in its CORS Access-Control-Allow-Headers. | | errors | boolean? | Capture uncaught browser errors (window.onerror + unhandledrejection) as logs. On by default — set false to disable. Independent of replay/tracing. | | captureConsole | 'error' \| 'warn' \| false | Capture console output as logs. 'error' (default) captures console.error; 'warn' captures console.warn and console.error; false disables it. Rate-capped + recursion-guarded. | | version | string? | Release identifier → the service.version resource attribute on exported browser logs (release attribution / suspect release). | | build | string? | Commit SHA → the service.build resource attribute on exported browser logs. |

Browser distributed tracing

By default /web only records session replay. Set tracing: true to also trace the browser: each outbound fetch becomes a CLIENT span, and the injected traceparent makes the downstream service's SERVER span its child — so a browser→API call renders as one connected trace and a service-map edge (web → api). This is separate from replay (it works even with replay off).

await instrumentWeb({
  apiKey: import.meta.env.VITE_HEYSTACK_API_KEY,
  service: "my-web-app",
  tracing: true,
  traceSampleRate: 0.25,                      // sample 25% of requests — tune for cost
  tracePropagationTargets: ["api.myapp.com"], // cross-origin backend(s) to trace through
});

Trace headers go to same-origin + your allow-list only — never third parties. The traceparent header is added only to same-origin requests and to URLs matching tracePropagationTargets. This is a safety guarantee: injecting traceparent on a cross-origin request forces a CORS preflight, and a third party that doesn't list the header (Clerk, Stripe, most APIs) rejects it — breaking that request. So calls to your own cross-origin backend need it added to tracePropagationTargets, and that backend must allow traceparent/tracestate in its CORS Access-Control-Allow-Headers. Same-origin backends need neither (no CORS involved).

Otherwise it's cost-aware and safe by design: off unless you opt in; head-sampled (an unsampled request still propagates traceparent with the sampled flag cleared, so the backend makes the same keep/drop decision — no orphaned server spans); and the exporter posts through the original fetch, never tracing its own upload (no self-export loop). Spans post to /v1/traces cross-origin with no CORS setup on your side.

Remote kill-switch (safety valve). Browser tracing can also be disabled from the server, without an app redeploy — a way to stop it fleet-wide if it ever misbehaves in the wild. It is fail-open: if the server config is unreachable or omits the flag, tracing stays on (absent flag = enabled), so the switch can only ever disable, never accidentally turn a working app off.

Browser error & console collection

instrumentWeb captures uncaught browser errors out of the box — no extra setup. Any window.onerror / unhandled promise rejection becomes an OTLP log (event.name=browser.error, ERROR severity) carrying the OTel exception.type / exception.message / exception.stacktrace semconv attributes, the page url.full, the session_id (so it correlates to the session replay), and — when tracing is on — the active trace_id / span_id. console.error is captured too by default.

await instrumentWeb({
  apiKey: import.meta.env.VITE_HEYSTACK_API_KEY,
  service: "my-web-app",
  version: "1.4.2",            // → service.version (release attribution)
  build: "abc1234",           // → service.build  (commit)
  captureConsole: "warn",     // capture console.warn + console.error (default: 'error')
  // errors: false,           // opt out of error capture entirely
});

It's the same cost-safe design as tracing/replay: logs POST to /v1/logs through the original fetch (captured before any patching) so the exporter never traces its own upload — no self-export loop. Console capture is recursion-guarded (anything logged on the export path is never re-captured) and rate-capped (max ~60 records/minute; the overflow is dropped and counted, with one summary log emitted when the cap lifts) so a runaway console.error in a render loop can't flood ingest. Errors show up in the console Logs tab and correlate to their session replay and trace.

Like tracing, error / console capture has a remote kill-switch: it can be disabled from the server without an app redeploy, and is fail-open (if the server config is unreachable or omits the flag, capture stays on — absent flag = enabled).

In-app bug reports (reportBug)

Let users report a bug from inside your app. reportBug is a headless API — no widget, you own the UX — that files a structured report and auto-attaches the context a triager needs: the current URL, user agent, replay session_id, active trace_id, release / build, and the last few captured browser errors (so you see what was going wrong on the page right before the report). The report also appears on the Logs timeline (event.name=user.bug_report).

Call it any time after instrumentWeb() has run (it throws if the SDK isn't initialised yet, or if message is empty). Network failures are swallowed — a failed report never breaks your app.

import { instrumentWeb, reportBug } from "@heystack/otel/web";

await instrumentWeb({ apiKey: import.meta.env.VITE_HEYSTACK_API_KEY, service: "my-web-app" });

// A minimal report button (wire this to whatever UI you like):
const btn = document.querySelector("#report-bug")!;
btn.addEventListener("click", async () => {
  const message = prompt("What went wrong?");
  if (!message) return;
  await reportBug({
    message,
    email: currentUser?.email,          // optional — so the team can follow up
    context: { plan: "pro", screen: "checkout" }, // optional app metadata (string→string)
  });
  alert("Thanks — your report was sent.");
});

Reports land in the console under the app's Bugs tab, already linked to the session replay, the trace, and the recent errors — one click from report to root cause. The POST goes to /v1/bug-reports via the same original fetch used by the other exporters (no self-export loop).

Session-replay keyframes (seekable playback)

The recorder checkpoints a fresh full snapshot roughly every 60 seconds and starts a new chunk at each one (a "keyframe"). The console player can therefore begin playback from the nearest keyframe at or before a target moment — e.g. jumping straight to where an error happened — instead of downloading the whole session first. No configuration; it's automatic.

Sampling & masking come from the console

By default sampling and masking are controlled from the Heystack console — enable replay and tune the sample rate / masking mode under Settings → Session replay for the app. sampleRate is only an optional local override; you don't need to pass it.

Privacy / masking (rrweb)

Masking is strict by default: all text inputs and passwords are masked before anything leaves the browser. For finer control, annotate your DOM:

| Attribute | Effect | | --- | --- | | data-hs-mask | Masks an element's text. | | data-hs-block | Blocks a region entirely (it's not recorded). | | data-hs-unmask | Reveals an element you explicitly trust (un-masks it). |

Flushing

On Workers/edge the export is a fetch() POST, and the isolate can be torn down the instant your handler returns. You must let that POST complete or the trace is silently dropped — this is the #1 cause of flaky Workers tracing. flushHeystack() and instrument()'s built-in flush both await the in-flight fetch (not just the OTel span processor, which does not wait for it).

• Standalone Workers (instrument()) — flushes automatically. After the response it ctx.waitUntils a promise that drains both the span processor and the exporter's in-flight fetch, so the POST finishes before the isolate is killed. No action needed.
• Next on Cloudflare/OpenNext (registerHeystack) — as of 0.3.0 this flushes automatically when @opennextjs/cloudflare is present: the export runs inside the Cloudflare request context, so the exporter borrows that request's ctx.waitUntil (via OpenNext's getCloudflareContext) to keep the isolate alive until the POST completes. No manual hook needed. For other workerd setups without @opennextjs/cloudflare, import { flushHeystack } from "@heystack/otel/workers" and call it from a response hook (or ctx.waitUntil(flushHeystack()) if you have a ctx) — or pass an explicit waitUntil to initHeystackWorkers (highest priority). flushHeystack() awaits the export fetch.
• Node (initHeystack) — flushes on SIGTERM/SIGINT automatically.

No feedback loop with host fetch auto-instrumentation

As of 0.3.1 the exporter suppresses tracing for its own ingest POST, and 0.3.2 makes that suppression actually take effect in production. On Next/OpenNext the host auto-instruments outbound fetch, so without this the exporter's POST /v1/traces became a CLIENT span → exported → re-captured → a sustained loop (wall-to-wall identical fetch POST .../v1/traces spans). The POST runs inside an OpenTelemetry tracing-suppressed context (suppressTracing) — but context.with() is a no-op unless a ContextManager is registered, which 0.3.1 did not do, so suppression silently did nothing. 0.3.2 registers a ContextManager (see Why a ContextManager), so the POST is genuinely suppressed.

As belt-and-suspenders the exporter also drops any span whose HTTP target points at the configured ingest origin. As of 0.3.2 that match is hostname-accurate: full-URL attributes (url.full, http.url) are parsed and compared on .hostname (case-insensitive, port-stripped) so a sibling domain like myingest.heystack.dev is no longer a false positive and an explicit port like ingest.heystack.dev:443 is correctly matched; host-only attributes (server.address, net.peer.name, net.peer.hostname, http.host, peer.address) are port-stripped and compared by hostname.

The /node (and Next-on-Node) path got the same guard in 0.4.3. Before then, only the Workers/Edge exporter dropped self-spans; on a plain Node runtime the OTLP-over-http exporter's own POST /v1/traces was still captured by the HTTP/undici auto-instrumentation and re-exported, so the same feedback loop ran on Node (production measurement: ~77% of all ingested spans were fetch POST .../v1/traces). 0.4.3 (a) configures the HTTP + undici auto-instrumentations to ignore outbound requests to the ingest host, and (b) wraps the exporter so any span targeting the ingest origin is filtered before export — the latter holds even if you pass your own instrumentations array. Same hostname-accurate matcher as the Workers path (shared module). No API change; upgrade and redeploy.

Migration / versioning

• 0.11.0 — /web: browser error + console collection, in-app bug reports, and seekable replay keyframes. instrumentWeb now captures uncaught errors (window.onerror + unhandledrejection) and console.error as OTLP logs by default (event.name=browser.error / browser.console, with exception.* semconv, url.full, session_id, and the active trace/span ids when tracing is on). New options: errors?: boolean (default true), captureConsole?: 'error' | 'warn' | false (default 'error'). New reportBug({ message, email?, context? }) API — a headless in-app bug reporter that POSTs to /v1/bug-reports and auto-attaches the URL, user agent, session_id, active trace_id, release/build, and the last ≤20 captured browser errors (also emitted as a user.bug_report log); reports triage in the console Bugs tab. The version / build options are now stamped as service.version / service.build on exported browser logs (release attribution). Console capture is recursion-guarded + rate-capped (no self-export loop, no flood). The recorder also checkpoints a keyframe (~60s) so the console player seeks from the nearest snapshot instead of loading the whole session. No breaking changes; all new options are optional.
• 0.10.0 — release / commit attribution (version + build) on every runtime entry. New optional options: version → the service.version resource attribute (a release identifier such as 1.4.2 or a git tag), build → the service.build resource attribute (the commit SHA). They power release health, suspect release, and suspect commit in the console — attributing a regression to the version/commit that introduced it (suspect-commit deep-links to the commit when a repo URL is configured for the app). Wired on /node, /next (threaded to whichever runtime runs), and /workers; accepted on /web for API symmetry. /node also reads HEYSTACK_SERVICE_VERSION / OTEL_SERVICE_VERSION and HEYSTACK_SERVICE_BUILD as env fallbacks. Both options are optional and only emitted when non-empty. No breaking changes. See Release & commit attribution.
• 0.9.2 — /workers: instrument() no longer breaks WebSocket / Durable Object upgrades. The fetch wrapper rebuilt every response with new Response(...), which drops the webSocket property on a 101 Switching Protocols upgrade — so every WebSocket connection on an instrumented Worker failed whenever tracing was active (an API key set), while passing in dev/staging where instrument() is a passthrough. instrument() now detects an upgrade (response.webSocket present, or status === 101) and returns the original response untouched (span closed immediately; no traceparent header injected — headers on a 101 aren't delivered anyway). Fixes failed wss:// connections behind routeAgentRequest / any new WebSocketPair() handler. No API change; upgrade and redeploy.
• 0.9.1 — /workers: binding spans no longer orphan without nodejs_compat. On a workerd runtime without nodejs_compat (no globalThis.AsyncLocalStorage), the synchronous fallback context manager cannot carry the active span across an await — so a D1/KV/R2/Vectorize/AI/Queue/Service span created after an await in your handler was emitted as the root of its own single-span trace instead of a child of the request. instrument() now captures the request's SERVER context per-request and uses it as an explicit parent fallback when no span is active, so binding operations are always children of the request trace. nodejs_compat is still recommended for cross-await parenting of outbound-fetch CLIENT spans and manual withSpan spans. No API change; upgrade and redeploy.
• 0.9.0 — /workers: automatic LLM gen_ai enrichment for outbound API calls. Outbound fetch calls to known LLM providers (OpenAI, Anthropic, Cloudflare AI Gateway, Google) automatically gain gen_ai.* OTel semantic-convention attributes on the CLIENT span — model, token counts, finish reason, response ID — with no extra code. New optional WorkersConfig.ai option: captureContent: true also captures prompt/completion text (off by default; strongly recommended for AI-app RCA), with redact for scrubbing and maxContentChars for length capping. The original request/response bodies are never consumed (request read only when already a string; response via response.clone()). Streaming responses skip response enrichment. No breaking changes.
• 0.8.0 — /workers: Workers AI, Queue producer, and Service binding instrumentation. instrumentBindings: true now auto-wraps three additional binding types: env.AI.run() emits CLIENT spans with gen_ai.system, gen_ai.request.model, and gen_ai.usage.input_tokens/output_tokens (streaming results are never consumed); Queue .send/.sendBatch emit PRODUCER spans with messaging.* attributes including batch size; Service binding .fetch emits a CLIENT span and injects a W3C traceparent header into the outgoing request so calls to other Workers appear in the same distributed trace. startSpan factory now accepts an optional SpanKind for correct CLIENT/PRODUCER categorisation. No breaking changes.
• 0.7.0 — /workers: remote sampling (sampling: { remote: true }). New sampling variant that fetches the head-sampling rate from the Heystack config endpoint at runtime, so you can change it from the console without redeploying. Cold isolates keep all traffic until the first config fetch resolves (fails open). If the config endpoint is unreachable, the worker keeps everything. Same parent-respecting rule as sampling: { rate }. No breaking changes; existing sampling: { rate } configs are unchanged.
• 0.6.0 — /workers: head sampling (sampling: { rate }). New optional WorkersConfig field: sampling.rate (0–1, default 1). Keeps a deterministic fraction of fresh root traces — the drop decision is made in the worker before export (no egress, no ingest cost). Parent-respecting: requests arriving with a sampled traceparent are always recorded. Consistent with server-side sampling (same trace-ID hash). No breaking changes; all new options are optional. See Head sampling.
• 0.5.0 — /workers: identity enrichment, binding tracing, outbound-fetch tracing, manual span helpers. New WorkersConfig options: getUser (attach enduser.id/session.id/http.request.id per request from a synchronous callback), instrumentBindings (auto child spans for D1/KV/R2/Vectorize — true = all detected, or a string[] to select). Outbound fetch calls made inside a traced handler automatically get CLIENT child spans with traceparent injection (distributed tracing across services). New ergonomic exports from /workers: withSpan(name, attrs?, fn) runs a function inside a named child span (auto-parented, exceptions recorded, span.end() in finally); addEvent(name, attrs?) adds an event to the active span. No breaking changes; all new options are optional.
• 0.4.3 — feedback-loop guard extended to the Node path (cost fix). The self-instrumentation loop the Workers path fixed in 0.3.1/0.3.2 was still live on plain Node / Next-on-Node: the OTLP-over-http exporter's POST /v1/traces was auto-instrumented and re-exported, so ~77% of ingested spans in production were the exporter tracing itself — needless ingest + ClickHouse compute. 0.4.3 ignores ingest-host requests in the HTTP/undici auto-instrumentations and filters self-spans at the exporter boundary (covers caller-supplied instrumentations too). The hostname matcher is now a shared module used by both /node and /workers. No API change. Action: upgrade and redeploy any Node/Next-on-Node app — it cuts ingested span volume sharply.
• 0.3.5 — type-constraint fix (Workers). A Worker whose queue consumer is typed with a concrete message body — queue(batch: MessageBatch<MyJob>, …), the normal case — failed to compile against instrument() in 0.3.4 (TS2345: MessageBatch<unknown> not assignable to MessageBatch<MyJob>). The WorkerHandler constraint declared its entrypoints as arrow properties, whose parameters are checked contravariantly under strictFunctionTypes, so a narrowed handler wasn't assignable. 0.3.5 declares them with method syntax (bivariant parameters) and widens the batch to MessageBatch<any>, mirroring Cloudflare's own ExportedHandler — a typed-queue Worker now type-checks with a bare instrument(handler, cfg). Runtime behaviour unchanged. Also adds a consumer type-check gate (pnpm consumer-typecheck, run in check and prepublishOnly) that compiles a fully-typed Worker against the built dist through the public exports map and asserts satisfies ExportedHandler<Env> — the regression that escaped in 0.3.3/0.3.4 now fails the build before publish.
• 0.3.4 — type-inference fix (Workers). Restores instrument()'s ability to infer the handler's concrete Env type. In 0.3.3 the signature was instrument<E = unknown, H extends WorkerHandler<E>>, so E defaulted to unknown and was never recovered from the handler — under strictFunctionTypes a Worker typed fetch(req, env: Env, ctx) then failed to compile (TS2345: 'Env' is not assignable to 'unknown') unless the caller passed instrument<Env>(...) explicitly. 0.3.4 infers E from the handler argument (instrument<H extends WorkerHandler<any>>(...): Instrumented<EnvOf<H>, H>), so a bare instrument(handler, cfg) type-checks again. Runtime behaviour is unchanged; no 0.3.3 consumer needs the explicit type arg after upgrading.
• 0.3.3 — /next uses bare, exports-mapped dynamic imports so the OpenNext (Cloudflare Pages) build resolves the workers entry correctly (fixes a build break). Workers/Node paths unchanged.
• 0.3.2 — runtime-correctness fixes:
  • ContextManager registered (CRITICAL). 0.3.1's suppressTracing was a no-op because no ContextManager was registered, so the exporter's ingest POST could still be re-traced into a feedback loop. 0.3.2 registers an AsyncLocalStorageContextManager (Node + workerd under nodejs_compat; sync stack-manager fallback otherwise) so suppression works — and as a bonus you get cross-await span parenting + per-request context isolation. Workers now require nodejs_compat. New dependency: @opentelemetry/context-async-hooks.
  • Hostname-accurate self-span filter (no sibling-domain false positives; host:port now matched; more host-only attrs covered).
  • OpenNext accessor race fixed — the getCloudflareContext accessor loads eagerly and early exports that fire before it resolves still get handed to ctx.waitUntil; accessor/waitUntil failures are now logged (once) instead of silently swallowed.
  • Node SDK hardening — initHeystack is idempotent (cached singleton) and registers SIGTERM/SIGINT handlers at most once (no leak across Next dev reloads); new optional instrumentations field to load a slimmer instrumentation set.
  • Drain timeout — instrument()'s ctx.waitUntil flush is raced against an ~8s timeout so a hung ingest can't pin the isolate to its CPU limit.
  • Bundler hints — /next's runtime-selected dynamic imports carry @vite-ignore + webpackIgnore so a Node build doesn't bundle the workers path (and vice-versa).
• 0.3.1 — instrument() now forwards queue/scheduled/tail (and any other handler) instead of returning only { fetch }, and traces queue/scheduled; the exporter suppresses self-tracing on its ingest POST (note: only effective from 0.3.2, which registers the ContextManager). Both are production-reproduced bug fixes; upgrade is recommended for any Queue/Cron Worker or Next-on-OpenNext app.
• Pin @heystack/otel >=0.3.0 — 0.3.0 makes Next-on-OpenNext auto-flush via the Cloudflare request context, hardens workerd detection (uses the WebSocketPair global so it survives nodejs_compat), and has instrument() set the global provider so nested spans export. The workerd-aware /next path and initHeystackWorkers / flushHeystack exports were added in 0.2.0.
• The pre-0.1.0 top-level default initHeystack({ apiKey }) is gone. Use the subpath entries: @heystack/otel/node, @heystack/otel/next, @heystack/otel/workers. The root @heystack/otel entry now exposes only pure helpers (buildExporterConfig, types).

Verify it's working

• Pass debug: true to initHeystack to print OTel diagnostic/export logs to the console.
• Short-lived processes: initHeystack registers SIGTERM/SIGINT handlers automatically, so the SDK flushes and shuts down before exit and you don't lose the last batch.

Then run your app and make a request — traces appear in your Heystack dashboard within seconds.