@stratchai/llm

Provider-agnostic forced-JSON LLM adapter for the stratchai trading agents. One interface, two providers — Anthropic (forced tool-use + prompt caching) and OpenRouter (OpenAI-style function calling) — with cache-aware per-call cost accounting so budget meters stay honest across vendors.

Built for decision gates that must never act on malformed output: the only operation offered is "produce an object matching this schema, or throw." Free-text JSON is deliberately not exposed — it empties/mis-formats in bursts on some models (~34% of live calls observed on our crypto gate), and a silently-defaulted decision is worse than a loud error.

Install

npm install @stratchai/llm

Zero runtime dependencies (built on fetch, Node ≥ 18).

Use

import { createProvider } from '@stratchai/llm'

const llm = createProvider({ model: process.env.GATE_MODEL }) // provider from LLM_PROVIDER (default anthropic)

const res = await llm.jsonCall({
  system: STATIC_RULEBOOK,          // byte-identical across calls → provider prompt-caching engages
  user: perCandidatePrompt,
  tool: {
    name: 'record_verdict',
    description: 'Record the entry-gate verdict.',
    schema: {
      type: 'object',
      properties: {
        take: { type: 'boolean' },
        reason: { type: 'string' },
        confidence: { type: 'number' },
      },
      required: ['take', 'reason', 'confidence'],
    },
  },
})
res.object        // schema-shaped verdict (never null — failure throws)
res.usage.costUsd // cache-aware $ for this call

Error contract (load-bearing)

• Transport/HTTP failures throw with .status set to the HTTP code.
• Parse/shape failures (missing tool call, bad JSON) throw with .status undefined.

Callers that retry only on e.status == null — the parse-only retry pattern our agents use, since HTTP-level retries belong to backoff logic — work unchanged across both providers. The recommended gate pattern is fail-closed: catch, log, and return a conservative skip.

Configuration

| env | meaning | default | |---|---|---| | LLM_PROVIDER | anthropic | openrouter | anthropic | | LLM_MODEL | model id in the provider's namespace | — (callers usually pass model) | | ANTHROPIC_API_KEY / OPENROUTER_API_KEY | credentials | — | | LLM_PRICE_IN_PER_MTOK / LLM_PRICE_OUT_PER_MTOK | $/1M-token overrides for cost metering | built-in table |

Built-in prices (USD per 1M tokens): Anthropic haiku 1/5, sonnet 3/15, opus 15/75 — cache reads at 0.1×, cache writes at 1.25×. OpenRouter: deepseek 0.3/1.2, llama-3.3-70b 0.2/0.6, qwen 0.3/1.2; unknown models fall back conservative (1/2) so a budget ceiling errs on the safe side — override the price envs when running anything else.

Provider notes

• Anthropic: tool_choice forces the verdict tool; the system block carries cache_control: ephemeral (disable per-call with cacheSystem: false).
• OpenRouter: function calling is the primary path. If the chosen model rejects tools (4xx mentioning tools), one fallback attempt uses response_format: json_object plus a schema-in-prompt instruction with first-{…} extraction. Prompt caching is automatic on supporting models (no marker to send); reported cached tokens are billed at the read discount.

Why this exists

Our agents are published to npm and must not be vendor-locked, but their decision gates carry a forward evidence record earned on a specific model. This package makes the provider a config choice while changing nothing about behavior on the default path — swap-safety is the point: promote a cheaper provider only after it earns the switch in a shadow A/B, not because the adapter made it easy.

Development

npm test   # tsc build + node:test (mocked fetch — no network, no keys)

Issues: https://github.com/stratchai/llm/issues