@evalguard/langchain

Drop-in EvalGuard instrumentation for LangChain and LangGraph in TypeScript / JavaScript.

Adds real-time firewall (2.57ms p95), 200+ eval scorers, 300+ attack plugins, and 50 compliance frameworks to any LangChain chain or LangGraph agent — without rewriting your code. One callback handler, full coverage.

Why this exists

LangChain users already have observability via LangSmith. This package adds the surface LangSmith doesn't ship:

• Real-time input + output guardrail enforcement (block prompt injection, PII exfiltration, jailbreaks in the hot path)
• 300+ plugin adaptive red teaming (run on demand from the dashboard against your chain)
• 50 compliance frameworks (EU AI Act Annex IV, ISO 42001 SoA, SOC 2 evidence collector)
• Vendor-neutral routing across 90+ first-party LLM providers + 800+ via aggregators (OpenRouter / LiteLLM / CometAPI)

You can run alongside LangSmith (both handlers in the same chain) or replace it by re-pointing your OTEL exporter to https://evalguard.ai/api/v1/ingest/otlp/traces.

Install

npm install @evalguard/langchain @langchain/core @langchain/openai

@langchain/openai is the chat model used by the examples below — swap it for @langchain/anthropic, @langchain/google-genai, or whichever provider package you actually use. Only @evalguard/langchain and @langchain/core are required by the wrapper itself.

⚠️ ESM-only — requires "type": "module"

@evalguard/langchain ships ES modules only ("type": "module", no CJS build). The quickstart below will not type-check or run in a default CommonJS TypeScript project — you get TS1479 ("the referenced file is an ECMAScript module and cannot be imported with require") and, because the peer SDK's types then resolve under a different module mode, a confusing TS2345 … Property '#private' … refers to a different member on the very first call.

To use this package, your consuming project must be ESM:

// package.json
{ "type": "module" }

// tsconfig.json
{ "compilerOptions": { "module": "node16", "moduleResolution": "node16" } }

Staying on CommonJS? Load it with a dynamic import(). Unlike the SDK-shaped wrappers (@evalguard/openai, @evalguard/anthropic, @evalguard/gemini), the handler takes no instance of a peer class across the boundary, and @langchain/core / @langchain/openai are dual-published, so your LangChain imports can stay static. Verified against @langchain/[email protected] + @langchain/[email protected].

The await must sit inside an async function: top-level await is an ESM-only feature, so a bare await import(...) at file scope in a CJS module is TS1309: The current file is a CommonJS module and cannot use 'await' at the top level.

// ✅ compiles under module/moduleResolution "node16", no "type": "module"
import { ChatOpenAI } from "@langchain/openai";

async function main() {
  const { EvalGuardCallbackHandler } = await import("@evalguard/langchain");
  const handler = new EvalGuardCallbackHandler({
    apiKey: process.env.EVALGUARD_API_KEY!,
    projectId: "proj-123",
  });

  const model = new ChatOpenAI({ model: "gpt-4o", callbacks: [handler] });
  return model;
}

void main();

If you swap in an LLM package that is itself ESM-only, that import has to become dynamic as well — the rule is that no ESM-only specifier may be imported statically from a CJS file, not just this one.

Node.js ≥ 22.12 can also require() an ESM module directly (require(esm)), but TypeScript still type-checks the import under CJS rules, so the dynamic-import form above is the supported path.

@langchain/core is a peer dependency — the wrapper compiles without it, so you can install lazily.

Usage

Per-model handler

import { ChatOpenAI } from "@langchain/openai";
import { EvalGuardCallbackHandler } from "@evalguard/langchain";

const handler = new EvalGuardCallbackHandler({
  apiKey: process.env.EVALGUARD_API_KEY!,
  projectId: "proj-123",        // optional
  blockOnViolation: true,        // default: true
  metadata: { env: "prod" },     // optional, attached to every trace
});

const model = new ChatOpenAI({
  model: "gpt-4o",
  callbacks: [handler],
});

const result = await model.invoke("Summarise this email…");

Global env-based install

import { autoInstall } from "@evalguard/langchain";

const handler = autoInstall();
// Reads EVALGUARD_API_KEY + EVALGUARD_PROJECT_ID + EVALGUARD_BASE_URL.
// Returns null if EVALGUARD_API_KEY is unset (safe to call at module load).

Then pass handler to any chain's callbacks array, or attach globally via setGlobalCallbacks([handler]) from @langchain/core/callbacks/manager.

LangGraph

The same handler covers LangGraph — node-level events (handleChainStart, handleToolStart, handleAgentAction) emit nested spans so the trace tree mirrors your graph topology.

LangGraph ships in its own package, so install it alongside the wrapper:

npm install @langchain/langgraph

A complete graph, start to finish. The handler goes in the RunnableConfig you pass to invoke, which is what makes every node in the graph — not just the top-level call — emit a span:

import { StateGraph, StateSchema, MessagesValue, START, END } from "@langchain/langgraph";
import { AIMessage, HumanMessage } from "@langchain/core/messages";
import { EvalGuardCallbackHandler } from "@evalguard/langchain";

const handler = new EvalGuardCallbackHandler({ apiKey: process.env.EVALGUARD_API_KEY! });

const State = new StateSchema({ messages: MessagesValue });

const app = new StateGraph(State)
  .addNode("respond", (state) => ({
    messages: [new AIMessage(`echo: ${state.messages.at(-1)?.text ?? ""}`)],
  }))
  .addEdge(START, "respond")
  .addEdge("respond", END)
  .compile();

const result = await app.invoke(
  { messages: [new HumanMessage("Summarise this email…")] },
  { callbacks: [handler] },
);

console.log(result.messages.at(-1)?.text);

Until 2026-08-03 this block referenced an undeclared workflow and wrote { messages: [...] } with a literal ... — TS1109: Expression expected. It never compiled, and LangGraph is a headline feature of this package. On 2026-08-03 the replacement above was type-checked and executed verbatim against @langchain/[email protected] + @langchain/[email protected], from the packed tarball in a clean ESM consumer, module/moduleResolution node16 + strict; it prints echo: Summarise this email… and emits three EvalGuard spans. That was a one-time check, not a CI gate — the langgraph:example anchors are there so the block can be extracted and re-run verbatim, which is the only way this stays true. Swap the respond node for a real chat model and the same handler covers its LLM span too.

Config

| Option | Default | Notes | |---|---|---| | apiKey | required | Your EvalGuard API key. | | projectId | none | Tags every trace with a project. Visible in the dashboard. | | baseUrl | https://evalguard.ai/api/v1 | Override for self-hosted deployments. | | blockOnViolation | true | When true, guardrail violations throw EvalguardBlockedError and abort the LangChain run. When false, violations are logged but the call proceeds. | | disableLogging | false | Skip trace logging entirely (guardrails still run). | | disableGuardrails | false | Skip pre-call firewall (tracing still runs). Setting this and blockOnViolation: true throws EvalguardConfigError at construction — with the firewall off there is nothing to block on, and silently accepting the pair would leave you believing you are protected. | | metadata | {} | Free-form fields attached to every trace. |

How blocking is actually enforced

LangChain swallows callback-handler exceptions by default, and by default does not even await the handler — so a handler that merely throws is a no-op and the model call proceeds. This handler therefore declares the two flags LangChain reads off the instance, awaitHandlers = true and raiseError = true, which is what makes the throw abort the run. Enforcement is verified end-to-end against a real @langchain/core chat model through invoke, stream and a RunnableSequence in src/__tests__/blocking-enforcement.test.ts.

Two consequences worth knowing:

• An unreadable prompt is rejected, not skipped. If the handler cannot extract the prompt text (for example a custom message class whose _getType() throws), it fails closed with a prompt_unreadable violation rather than letting an unscanned prompt reach the model. With blockOnViolation: false the call proceeds and the trace records that nothing was scanned. The prompt is never partially scanned or truncated.
• Only the guardrail decision can abort your run. Model-name resolution degrades to "unknown", and every trace-logging hook is contained, so a logging failure never bubbles into your chain.

What's traced

Every LangChain run emits to EvalGuard with:

• LLM spans — input, output, model, latency, token usage, cost (priced via the same table as openai-wrapper / anthropic-wrapper)
• Chain spans — Runnable + LangGraph node entry/exit
• Tool spans — every Tool.invoke or LangGraph tool node
• Agent actions — chosen tool + input for each agent step

All spans carry OpenInference-compliant attributes (openinference.span.kind, llm.model_name, llm.token_count.*, input.value, output.value, status.code). This means the same traces render natively in Phoenix, Arize AX, and any OpenInference-aware viewer.

Outage semantics — fail-CLOSED by default

Corrected 2026-07-29. This section previously documented a fail-open contract ("lets the LangChain call proceed — guardrails default-allow on error"). That has not been true since 2026-05-28, when the wrappers moved to fail-closed. Customers reading this were told a security control degrades safely when it does the opposite.

If EvalGuard is unreachable (network blip, our outage, DNS hiccup), this wrapper's behaviour depends on blockOnViolation:

| blockOnViolation | Guardrail unreachable | Real violation detected | |---|---|---| | true (default) | Blocks — throws EvalguardBlockedError with a guardrail_unavailable violation | Blocks — throws EvalguardBlockedError | | false | Call proceeds, violation recorded on the trace | Call proceeds, violation recorded on the trace |

Trace logging always fails silently either way — a trace-log failure never bubbles into your chain.

If you need availability-first behaviour, set blockOnViolation: false. The legacy global escape hatch EVALGUARD_GUARDRAIL_FAIL_OPEN_LEGACY=1 restores pre-2026-05-28 fail-open for every wrapper in the process; it prints a one-time warning and is not recommended.

Same contract as our openai-wrapper, anthropic-wrapper, gemini-wrapper, vercel-ai-wrapper, and llamaindex-wrapper.

Blocking errors

When blockOnViolation: true (default), policy violations throw EvalguardBlockedError:

import { EvalGuardCallbackHandler, EvalguardBlockedError } from "@evalguard/langchain";

try {
  const result = await chain.invoke({ input: userMessage });
} catch (err) {
  if (err instanceof EvalguardBlockedError) {
    console.log("Blocked:", err.violations);
    // err.violations: [{ type, severity, message }]
    return "I can't help with that.";
  }
  throw err;
}

Migrating from LangSmith

Already on LangSmith? Two paths:

Coexist (recommended for evaluating): Keep your existing LangChainTracer in the callbacks array; add EvalGuardCallbackHandler alongside. Both will receive the same events; LangSmith continues to populate its dashboard while EvalGuard adds firewall + red-teaming + compliance.

Replace: Remove the LangSmith tracer. Re-point your OTEL exporter:

# Before
OTEL_EXPORTER_OTLP_ENDPOINT=https://api.smith.langchain.com/otel/

# After (full traces path — use the signal-specific env var)
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://evalguard.ai/api/v1/ingest/otlp/traces
OTEL_EXPORTER_OTLP_HEADERS=Authorization=Bearer%20${EVALGUARD_API_KEY}

We accept OpenInference-shaped spans natively, so traces emitted via LangChain's existing OTel integration continue to flow.

Cost estimates: unpriced models are reported as unpriced

estimateCost() used to invent a price for any model outside a ~60-row table: estimateCost("gpt-5", 1000, 500) and estimateCost("totally-unknown-model", 1000, 500) both returned 0.0105 from a blended $0.003/$0.015-per-1k fallback — with no flag and no warning. A FinOps figure you cannot tell apart from a real vendor price is worse than no figure at all.

Two things changed:

• Coverage — 2,200+ model ids now resolve to a real, sourced rate (generated from EvalGuard's own pricing database, which is synced from the LiteLLM catalogue). gpt-5 is priced correctly.
• Honesty — a genuinely unknown model is now visibly unknown.

import { estimateCostDetailed, isModelPriced } from "@evalguard/langchain";

estimateCostDetailed("gpt-5", 1000, 500);
// { model: "gpt-5", costUsd: 0.00625, priced: true, pricingSource: "catalog" }

estimateCostDetailed("totally-unknown-model", 1000, 500);
// { model: "totally-unknown-model",
//   costUsd: null,              // <- never a fabricated number
//   priced: false,
//   pricingSource: "unpriced",
//   blendedFallbackUsd: 0.0105 } // <- opt-in rough figure, clearly labelled

isModelPriced("totally-unknown-model"); // false

estimateCost() still returns a number for backwards compatibility, but it now emits a one-time console.warn naming the unpriced model. Traces carry costPricingSource alongside cost, and cost is null for an unpriced model rather than a guess, so your EvalGuard dashboard shows "unpriced" instead of a fake dollar amount.

License

Apache-2.0