npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2026 – Pkg Stats / Ryan Hefner

@statsig/statsig-ai

v1.0.0

Published

Statsig is the on-stop data platform for product development. We help you move faster with feature gates (feature flags), experiments, and the world's most powerful stats engine. If you're new to Statsig, check out our product and create an account at [st

Readme

Statsig Node AI SDK

Statsig is the on-stop data platform for product development. We help you move faster with feature gates (feature flags), experiments, and the world's most powerful stats engine. If you're new to Statsig, check out our product and create an account at statsig.com.

The AI SDK provides convenient hooks for developers working with LLMs to iterate on their promps, run evals to test their efficacy, add observability to LLM usage.

Getting Started

Read through the Documentation to see code samples and how to get started.

Tracing model calls

Wrap your provider client to record each model call as an OpenTelemetry span. Calls made inside prompt.withLive(...) are tagged with the prompt name, version name and version ID.

import { wrapAnthropic, wrapGemini, wrapOpenAI } from '@statsig/statsig-ai';

const openai = wrapOpenAI(new OpenAI());
const anthropic = wrapAnthropic(new Anthropic()); // messages.create (incl. stream: true) and messages.stream()
const gemini = wrapGemini(new GoogleGenAI({})); // models.generateContent / generateContentStream

Messages are only recorded when you opt in with captureOptions (for example { captureOptions: { capture_all: true } }). All wrappers record gen_ai.input.messages, gen_ai.output.messages and gen_ai.system_instructions in the OpenTelemetry GenAI message format (role + parts such as text, tool_call and tool_call_response).

Tagging calls with a prompt version

Run the code that renders and sends a prompt inside prompt.withLive(...), or prompt.withVersion(version, ...) for a Candidate. Wrapped model calls and wrap spans inside it get statsig.llm.prompt_name, statsig.llm.prompt_version and statsig.llm.prompt_version_id, which Statsig uses to attribute online eval grades to a version.

The variables you pass to getPromptMessages are recorded as statsig.llm.prompt_template_variables (JSON), so the request can be re-run on another version. Model call spans only record them when input capture is on (capture_all or capture_input_messages). Workflow spans record the variables rendered inside them, so start the workflow inside the version scope.

const prompt = ai.getPrompt(user, 'support_bot');

const runTurn = wrap({ type: 'workflow', name: 'support_turn' }, (version) =>
  openai.chat.completions.create({
    model: version.getModel(),
    messages: version.getPromptMessages({ input }),
  }),
);

await prompt.withLive(runTurn);

Marking the environment

The environment option of initializeTracing (defaults to NODE_ENV) is recorded on the resource as deployment.environment.name. To tag only some spans, for example evals running in the same process as production code, run that code inside withStatsigEnvironment. Spans started inside it get a statsig.environment attribute, which overrides the resource value for those spans.

Use production for live traffic. eval and online_eval_judge are reserved for spans Statsig generates: Eval() runs its tasks and scorers with statsig.environment=eval.

initializeTracing({ environment: 'production' });

await withStatsigEnvironment('staging_canary', () => handleRequest(req));

Tracing evals

When tracing is initialized, Eval() gives each record its own trace. The root span is a workflow span marked statsig.environment=eval, with statsig.eval.name, statsig.eval.run_name and statsig.eval.record_index. LLM calls the task makes through wrapOpenAI, wrapAnthropic or wrapGemini are its children, and each scorer runs in its own child span. Each result's traceID and spanID are sent to Statsig with the scores, and Eval() flushes the spans before it returns.

Joining a Statsig eval trace

A workflow eval sends each dataset row to your endpoint as a POST request. To show your agent's tool and model calls under the eval row in the trace drawer, run the handler inside withIncomingTraceContext. It reads the W3C traceparent and tracestate request headers, so wrapped model calls and wrap spans become children of the eval row's span. It accepts Node request headers (req.headers) or a Fetch Headers object. If there's no valid traceparent, the handler runs as usual.

import { withIncomingTraceContext } from '@statsig/statsig-ai';

// Node / Express
app.post('/agent', (req, res) =>
  withIncomingTraceContext(req.headers, async () => {
    res.json({ output: await runAgent(req.body) });
  }),
);

// Fetch-style handlers (Next.js route handlers, Hono, ...)
export async function POST(request: Request) {
  return withIncomingTraceContext(request.headers, async () =>
    Response.json({ output: await runAgent(await request.json()) }),
  );
}

If your server already uses OpenTelemetry HTTP instrumentation, its spans join the trace on their own. The helper keeps the active span when it's already in the incoming trace, so using both is safe.

What Statsig sends: workflow eval requests carry Content-Type: application/json, the headers you configured for the workflow, and two headers of their own. traceparent names the eval row's root span, and x-statsig-eval-run holds the eval run ID. A header you configure with the same name wins, matched case-insensitively. Test Connection doesn't send either header.

Spans from your endpoint get your app's environment, so eval requests land with your production traces, where online evals can grade them. To keep them out, check for x-statsig-eval-run and run the handler inside withStatsigEnvironment('eval', ...). You can use the same check to skip side effects.

import {
  withIncomingTraceContext,
  withStatsigEnvironment,
} from '@statsig/statsig-ai';

app.post('/agent', (req, res) =>
  withIncomingTraceContext(req.headers, () => {
    const handle = async () => {
      res.json({ output: await runAgent(req.body) });
    };
    return req.headers['x-statsig-eval-run']
      ? withStatsigEnvironment('eval', handle)
      : handle();
  }),
);

Running evals on a Statsig dataset

Pass statsigDataset(name, toRecord) as Eval()'s data to run on an eval dataset from the console. toRecord says which columns hold the input and the expected value. The run names the dataset and each result keeps its dataset row id, so the Eval page compares runs on the dataset row by row.

import { Eval, statsigDataset } from '@statsig/statsig-ai';

await Eval('support_bot_quality', {
  data: statsigDataset('support_cases', (row) => ({
    input: row.columns.question,
    expected: row.columns.answer,
    category: row.groups,
  })),
  task: (input) => answer(input),
  scorer: ({ output, expected }) => output === expected,
});

To post your own records for a dataset instead, set dataset and give each record the rowID of its dataset row.

Latency and tokens

Eval() times each record's task, from the call until its output is ready, and sends that with the result as latencyMs. Scorers aren't part of it.

The runner can't see your model calls, so the task reports its tokens with reportTokens from its second argument. Each call adds to the record's totals, so a task that makes several model calls can report each one, or its total once. The totals are sent as inputTokens and outputTokens.

await Eval('support_bot_quality', {
  data,
  task: async (input, { reportTokens }) => {
    const { choices, usage } = await openai.chat.completions.create({
      model: 'gpt-4o-mini',
      messages: [{ role: 'user', content: input }],
    });
    if (usage) {
      reportTokens({
        inputTokens: usage.prompt_tokens,
        outputTokens: usage.completion_tokens,
      });
    }
    return choices[0].message.content ?? '';
  },
  scorer: ({ output, expected }) => output === expected,
});

The run page shows a Latency column, and a Tokens column when the task reported tokens. Neither is graded. Eval() also returns them on each result. A record whose task throws sends neither, and a record whose task never calls reportTokens sends no tokens. reportTokens ignores a call whose counts aren't finite numbers of at least 0, and logs a warning.

Community

If you need any assistance or just have a question, feel free to reach out to us on Slack.