@tokz/langchain

Drop-in tokz compression for LangChain chat models. Wrap any BaseChatModel; ToolMessage content is compressed through the tokz API before every invoke, stream, and batch. No other code changes.

Install

npm install @tokz/langchain @tokz/sdk @langchain/core

Usage

import { ChatOpenAI } from "@langchain/openai";
import { ToolMessage } from "@langchain/core/messages";
import { withTokz } from "@tokz/langchain";

const model = withTokz(new ChatOpenAI({ model: "gpt-4o" }), {
  apiKey: process.env.TOKZ_API_KEY, // required
});

// ToolMessage content is compressed automatically before the model call.
await model.invoke([
  new HumanMessage("How many pods are running?"),
  new ToolMessage({ content: bigKubectlJson, tool_call_id: "call_1" }), // ← compressed
]);

Before / after

A 4,000-token tool result, at the default targetRatio: 0.45:

| | tokens | |---|---| | before | ~4,000 | | after | ~1,800 |

Only ToolMessage is compressed. response_metadata, usage_metadata, id, tool_call_id, and name are preserved; the message stays a ToolMessage. Every emitted span is a byte-exact substring of your original text.

Options

withTokz(model, {
  apiKey: string,        // required
  baseUrl?: string,      // default https://api.tokz.dev
  targetRatio?: number,  // default 0.45 (fraction of bytes kept)
  onSavings?: (originalTokens, compressedTokens) => void,
});

Token counts in onSavings are estimates (~4 bytes/token).

Docs

Full documentation: https://tokz.dev/docs