@evalguard/vercel-ai

Drop-in Vercel AI SDK middleware that adds EvalGuard guardrails (firewall input checks) and observability (trace logging, cost tracking) to any language model from @ai-sdk/openai, @ai-sdk/anthropic, @ai-sdk/google, @ai-sdk/groq, or any other provider that follows the AI SDK model interface.

Verified against ai@5, ai@6 and ai@7 (model spec v2 / v3 / v4) — by a check that runs before this package is published, not by assertion. For each major, scripts/verify-vercel-ai-peer-matrix.mjs installs the packed tarball into a clean consumer alongside that version of ai and its matching @ai-sdk/openai, type-checks the quickstart below verbatim against it, and executes withEvalguard end to end through that version's own generateText / streamText (against a stub model, no network), asserting the emitted trace carries numeric token counts.

The exact versions covered are in peer-matrix.json, which ships in this tarball. Two steps in .github/workflows/publish-wrappers.yml run before this package is published — pnpm gate:publish-readiness and pnpm gate:vercel-ai-peers — and the first fails the release if the paragraph above and peer-matrix.json disagree in either direction, or if no workflow runs the second. So the sentence above cannot outlive the thing that makes it true. (Naming both commands is deliberate: until 2026-08-03 this paragraph said "a CI gate" and gate:publish-readiness appeared in no workflow at all, which made the sentence describing the gate the very kind of unbacked claim the gate exists to catch.)

The wrapper is a Proxy over the underlying model that intercepts only doGenerate / doStream; every other property read is forwarded to the real instance. That matters because real provider models are class instances whose supportedUrls / provider / modelId are prototype getters — an object spread ({ ...model }) copies own enumerable properties only and would drop all of them. Capability fields therefore pass through untouched and URL/file inputs the model handles natively are not stripped or re-downloaded.

Output text is read from the content array. Token counts are read from usage in whichever shape the peer uses: a flat inputTokens number on ai@5, the { total, … } breakdown object on ai@6 / ai@7, and the legacy v1 promptTokens / completionTokens as a back-compat fallback.

Install

npm install @evalguard/vercel-ai ai @ai-sdk/openai

ai is a peer dependency (npm installs it for you, but naming it pins the major you want). @ai-sdk/openai is the provider used by the quickstart below — swap it for @ai-sdk/anthropic, @ai-sdk/google, or whichever provider you actually use.

⚠️ ESM-only — requires "type": "module"

@evalguard/vercel-ai ships ES modules only ("type": "module", no CJS build). The quickstart below will not type-check or run in a default CommonJS TypeScript project — you get TS1479 ("the referenced file is an ECMAScript module and cannot be imported with require") and, because the peer SDK's types then resolve under a different module mode, a confusing TS2345 … Property '#private' … refers to a different member on the very first call.

To use this package, your consuming project must be ESM:

// package.json
{ "type": "module" }

// tsconfig.json
{ "compilerOptions": { "module": "node16", "moduleResolution": "node16" } }

Staying on CommonJS? Load it with a dynamic import(). Unlike the SDK-shaped wrappers (@evalguard/openai, @evalguard/anthropic, @evalguard/gemini), ai and the @ai-sdk/* providers are dual-published, so those imports can stay static. Verified against ai@5 + @ai-sdk/openai@2.

The await must sit inside an async function: top-level await is an ESM-only feature, so a bare await import(...) at file scope in a CJS module is TS1309: The current file is a CommonJS module and cannot use 'await' at the top level.

// ✅ compiles under module/moduleResolution "node16", no "type": "module"
import { openai } from "@ai-sdk/openai";

async function main() {
  const { withEvalguard } = await import("@evalguard/vercel-ai");
  const model = withEvalguard(openai("gpt-4o-mini"), {
    apiKey: process.env.EVALGUARD_API_KEY!,
    projectId: "proj-123",
  });

  // …then pass `model` to generateText / streamText as the quickstart does.
  return model;
}

void main();

The export is withEvalguard — lower-case g. This block said withEvalGuard until 2026-08-01, which does not exist and fails with TS2339: Property 'withEvalGuard' does not exist.

If you swap in a provider package that is itself ESM-only, that import has to become dynamic as well — the rule is that no ESM-only specifier may be imported statically from a CJS file, not just this one.

Node.js ≥ 22.12 can also require() an ESM module directly (require(esm)), but TypeScript still type-checks the import under CJS rules, so the dynamic-import form above is the supported path.

Peer requires ai >= 5.0.0 (ai@5, ai@6 and ai@7 are all verified).

Use

import { openai } from "@ai-sdk/openai";
import { generateText } from "ai";
import { withEvalguard } from "@evalguard/vercel-ai";

const model = withEvalguard(openai("gpt-4o-mini"), {
  apiKey: process.env.EVALGUARD_API_KEY!,
  projectId: "proj-123",
});

const { text } = await generateText({ model, prompt: "Say hi." });

The two comments above are load-bearing: scripts/verify-vercel-ai-peer-matrix.mjs extracts exactly what is between them and type-checks that text against every peer major, so this block cannot drift from what is actually verified. Removing them fails the gate.

That's it — every call now:

1. Runs your prompt through EvalGuard's 5-layer firewall (pattern, token, semantic, output validation, allow-list precedence) before reaching the LLM. Blocks on violation by default — set blockOnViolation: false to log-only.
2. Records a trace with model, provider, input, output, latency, token usage, and estimated cost to your EvalGuard project.

Options

withEvalguard(model, {
  apiKey: "...",
  projectId: "proj-...",        // optional — required to scope traces
  baseUrl: "https://...",        // optional — self-hosted EvalGuard
  blockOnViolation: true,        // default: true
  disableGuardrails: false,      // default: false
  disableLogging: false,         // default: false
  metadata: { feature: "support-bot", env: "prod" },
});

Streaming works too

Stream parts pass through verbatim. The trace is logged once on finish with the assembled text and token totals.

import { streamText } from "ai";

const { textStream } = await streamText({ model, prompt: "..." });
for await (const chunk of textStream) {
  process.stdout.write(chunk);
}

Outage semantics — fail-CLOSED by default

Corrected 2026-07-29. This section previously promised a fail-open "guarantee" ("falls back to allow + don't log"). The wrappers went fail-closed on 2026-05-28; the docs were never updated.

If EvalGuard's API is unreachable, slow, or returns an error:

• blockOnViolation: true (default) — the wrapper throws EvalguardBlockedError with a guardrail_unavailable violation. The provider call is not made.
• blockOnViolation: false — the call proceeds and the outage is recorded.

Trace-log errors are silently swallowed in both modes.

EvalguardBlockedError is also what a real firewall block raises, and it is catchable distinctly:

import { EvalguardBlockedError } from "@evalguard/vercel-ai";

try {
  const { text } = await generateText({ model, prompt: userInput });
} catch (err) {
  if (err instanceof EvalguardBlockedError) {
    return { error: "blocked", violations: err.violations };
  }
  throw err;
}

Cost estimates: unpriced models are reported as unpriced

estimateCost() used to invent a price for any model outside a ~60-row table: estimateCost("gpt-5", 1000, 500) and estimateCost("totally-unknown-model", 1000, 500) both returned 0.0105 from a blended $0.003/$0.015-per-1k fallback — with no flag and no warning. A FinOps figure you cannot tell apart from a real vendor price is worse than no figure at all.

Two things changed:

• Coverage — 2,200+ model ids now resolve to a real, sourced rate (generated from EvalGuard's own pricing database, which is synced from the LiteLLM catalogue). gpt-5 is priced correctly.
• Honesty — a genuinely unknown model is now visibly unknown.

import { estimateCostDetailed, isModelPriced } from "@evalguard/vercel-ai";

estimateCostDetailed("gpt-5", 1000, 500);
// { model: "gpt-5", costUsd: 0.00625, priced: true, pricingSource: "catalog" }

estimateCostDetailed("totally-unknown-model", 1000, 500);
// { model: "totally-unknown-model",
//   costUsd: null,              // <- never a fabricated number
//   priced: false,
//   pricingSource: "unpriced",
//   blendedFallbackUsd: 0.0105 } // <- opt-in rough figure, clearly labelled

isModelPriced("totally-unknown-model"); // false

estimateCost() still returns a number for backwards compatibility, but it now emits a one-time console.warn naming the unpriced model. Traces carry costPricingSource alongside cost, and cost is null for an unpriced model rather than a guess, so your EvalGuard dashboard shows "unpriced" instead of a fake dollar amount.

License

Apache-2.0