inferock-bench

Local benchmark proxy for OpenAI, Anthropic, Gemini, and pinned OpenRouter calls. Provider keys are not sent to Inferock; attached only to provider requests. It accepts a generated local ibl_ bench key from your SDK and writes per-call receipts with spent dollars, bill-bounded money loss, time loss, provider-recognized recovery, recognition gap, and invoice-check exposure kept separate.

Use it when you want to audit an AI/LLM bill, measure Claude or GPT token usage locally, or answer whether a failed API call was still billed. It helps investigate OpenAI overcharging-token questions and Anthropic billing-error questions by preserving local token usage, cost, retry, and failure evidence without sending provider keys to a hosted service.

Full docs, screenshots, and the public standard live at https://github.com/inferock/inferock-bench.

| Need | Start here | | --- | --- | | Run the local proxy | Quickstart | | See command surface | Commands | | Understand the trust boundary | What leaves your machine | | Read the receipt rulebook | The Inferock Standard |

Quickstart

npx inferock-bench

The running CLI prints a line like:

Dashboard: http://127.0.0.1:4318/?token=<one-time-token>

Open the exact token-bearing Dashboard: URL printed by that process, save your provider key locally, then copy the local bench key into your SDK config with the local base URL. The real URL contains a newly generated token; do not remove its ?token=... query or substitute the tokenless root.

OpenAI:

const openai = new OpenAI({
  apiKey: process.env.INFEROCK_BENCH_KEY ?? "ibl_your_generated_local_key",
  baseURL: "http://127.0.0.1:4318/v1",
});

Anthropic:

const anthropic = new Anthropic({
  apiKey: process.env.INFEROCK_BENCH_KEY ?? "ibl_your_generated_local_key",
  baseURL: "http://127.0.0.1:4318",
});

Gemini:

await fetch("http://127.0.0.1:4318/v1beta/models/gemini-2.5-flash:generateContent", {
  method: "POST",
  headers: {
    "content-type": "application/json",
    "x-api-key": process.env.INFEROCK_BENCH_KEY ?? "ibl_your_generated_local_key",
  },
  body: JSON.stringify({
    contents: [{ parts: [{ text: "Write one sentence about local receipts." }] }],
  }),
});

OpenRouter:

const openrouter = new OpenAI({
  apiKey: process.env.INFEROCK_BENCH_KEY ?? "ibl_your_generated_local_key",
  baseURL: "http://127.0.0.1:4318/openrouter/v1",
});

The terminal prints first call measured ✓ after the first successful proxied call.

inferock-bench init --patch path/to/client.ts --yes can patch simple new OpenAI({ ... }) and new Anthropic({ ... }) constructors. It updates both apiKey and baseURL; if it cannot do that safely, it refuses and tells you what to change by hand. Gemini and OpenRouter setup are supported through inferock-bench setup, the dashboard, and their local proxy paths.

Commands

inferock-bench start
inferock-bench start --host 0.0.0.0 --allow-external-host
inferock-bench init
inferock-bench test --providers all
inferock-bench test --generator agent
inferock-bench report --last 24h
inferock-bench receipt --compact
inferock-bench telemetry enable --reliability-index

The default server host is 127.0.0.1. Non-loopback hosts are refused unless --allow-external-host is present; that mode prints a warning because the proxy and management APIs become reachable from other machines that can connect to the host.

inferock-bench test shows the full-battery estimated price before any provider call. Agent mode auto-provisions pinned [email protected] locally only after explicit consent, or uses --agent-cmd as a user-supplied agent.

Production routing, secure key custody, invoice reconciliation, provider leverage, and audit workflows belong in hosted Inferock: https://inferock.opiusai.com.

License

inferock-bench is source-available under FSL-1.1-ALv2 and converts to Apache-2.0 after two years. The full current terms are in LICENSE. Earlier releases had incomplete license files or legacy metadata; see the historical package license advisory and upgrade to 0.2.4 or later.

Founders

Built by Bharath Koneti (LinkedIn, X) and Himashwetha Gowda (LinkedIn), building Inferock at OpiusAI — the AI-provider accountability standard.

What to read next

• Root README for screenshots, the public run card, and the docs map.
• Key handling before saving provider credentials.
• Coverage test methodology before running the complete test battery.