@virajmishra1/bench-anthropic

Auto-instrument the Anthropic SDK with Bench. One line.

npm install @virajmishra1/bench-anthropic @virajmishra1/bench-sdk @anthropic-ai/sdk

Use it

import Anthropic from "@anthropic-ai/sdk";
import { observe } from "@virajmishra1/bench-sdk";
import { wrapAnthropic } from "@virajmishra1/bench-anthropic";

const bench = observe({
  apiKey: process.env.BENCH_KEY,
  agent: "my-agent",
});

const client = wrapAnthropic(new Anthropic(), bench);

// Every call becomes a Bench task with a versioned API-cost estimate.
const result = await client.messages.create({
  model: "claude-sonnet-4-6",
  max_tokens: 1024,
  messages: [{ role: "user", content: "what's 2+2?" }],
});

That's it. Open bench.virajmishratakehome.workers.dev/dashboard and watch your messages stream in with per-call estimated API cost, latency, and LLM-judge scores.

What gets recorded

For each messages.create (and completions.create):

| Bench field | Source | |---|---| | task.name | "messages.create (claude-sonnet-4-6)" (configurable) | | task.input | The messages + model + tool config (truncated if huge) | | task.output | The full Anthropic response | | task.duration_ms | Wall time end-to-end | | task.cost_usd | Estimated from usage × versioned Anthropic list pricing | | task.cost_source | estimated (never presented as a provider invoice) | | task.cost_breakdown | Model, token/cache counts, pricing version and source URL | | task.status | success or failed (auto on throw) |

Options

wrapAnthropic(client, bench, {
  priceTable: { ... },        // override current standard/global list pricing
  taskName: (m, p) => "...",  // custom task names
  captureInput: false,         // skip recording inputs (PII / privacy)
  captureOutput: false,        // skip recording outputs
});

Bench distinguishes estimates from owner-entered and provider-reported cost. Contract discounts, subscription allowances, server-tool charges, and other provider adjustments can differ from the API-equivalent estimate.

License

MIT