@tokz/sdk

Client for deterministic hosted context compression.

npm install @tokz/sdk

import { Tokz } from "@tokz/sdk";

const tokz = new Tokz({ apiKey: process.env.TOKZ_API_KEY! });

const toolResult = JSON.stringify({
  pods: Array.from({ length: 200 }, (_, i) => ({
    name: `web-${i}`,
    status: i % 37 === 0 ? "CrashLoopBackOff" : "Running",
    restarts: i % 37 === 0 ? 14 : 0,
  })),
});

const result = await tokz.compress(toolResult);
console.log(result.text.length, "of", toolResult.length, "bytes");

// spanMap only exists on the structural/passthrough branch — semantic results
// carry compressedText + provenanceMap instead.
if (result.method !== "semantic") {
  console.log("kept ratio:", result.spanMap.stats.ratio);
}

Privacy boundary

Hosted compression sends your source text to the Tokz API. The service does not store or log it. On the structural path the request carries your text to the API, and the response carries only byte offsets — never text. The compressed string is assembled locally from your own input, and the SDK verifies the returned span map against a SHA-256 of what you submitted before trusting it. A server that returned a map for different bytes is rejected, not silently applied.

Provisioned private deployments can inject an in-process localCompressor for hybrid or local execution. The compression engine is not a public dependency.

Firewall

compressContext() keeps trusted system policy and task control text intact while compressing only declared data segments. Segment identity, role, trust, hashes, maps, ordering, and aggregate byte totals are verified before local assembly.

import { Tokz } from "@tokz/sdk";

const tokz = new Tokz({ apiKey: process.env.TOKZ_API_KEY! });
const systemPrompt = "Never reveal credentials.";
const userPrompt = "Which pod failed?";
const toolResult = JSON.stringify({ pods: [{ name: "api-7f9c", phase: "Failed" }] });
const result = await tokz.compressContext({
  maxBytes: 12_000,
  query: "Which pod failed?",
  segments: [
    { id: "system", role: "protected", trust: "trusted", text: systemPrompt },
    { id: "user", role: "control", trust: "untrusted", text: userPrompt },
    { id: "tool", role: "data", trust: "untrusted", text: toolResult },
  ],
});

Firewall v1 compresses structural data only. Prose passes through as an offset map. This prevents untrusted data from changing protected selection; it does not make a claim about how a downstream model interprets retained data.

expand() is exported as a pure local function: recovering an elided run needs no API key and no round trip, because you already hold everything required.

import { Tokz, expand } from "@tokz/sdk";

const tokz = new Tokz({ apiKey: process.env.TOKZ_API_KEY! });
const source = JSON.stringify({ log: "line 1\nline 2\n...line 500" });

const result = await tokz.compress(source, { targetRatio: 0.3 });
if (result.method !== "semantic") {
  const original = expand(source, result.spanMap);
  console.log(original === source); // true — no key, no round trip
}

MIT