ai-cruz-tokens

Cost-optimized LLM execution engine for TypeScript. Drop-in middleware that wraps LLM SDKs and automatically minimizes cost per request.

Features

• Smart Router — Content-aware model selection based on message complexity
• Prompt Stratifier — Automatic cache control for system prompts (Anthropic ephemeral caching)
• Budget Guard — Real-time spend tracking with automatic model degradation
• Pre-fetch Pipeline — Bypass the LLM for deterministic tool workflows
• History Optimizer — Sliding window trimming that preserves tool call/result pairs
• Multi-provider — Anthropic, OpenAI, and Google Gemini adapters

Install

npm install ai-cruz-tokens

Provider SDKs are peer dependencies — install only what you use:

npm install @anthropic-ai/sdk    # for Claude models
npm install openai               # for GPT models
npm install @google/generative-ai # for Gemini models

Quick Start

import { aiCruzTokens } from "ai-cruz-tokens";

const client = aiCruzTokens({
  models: [
    { id: "claude-sonnet-4", provider: "anthropic", tier: "premium",
      inputCostPer1M: 3, outputCostPer1M: 15, cacheCostPer1M: 3.75,
      cacheReadPer1M: 0.3, supportsCaching: true, supportsTools: true },
    { id: "claude-haiku-4", provider: "anthropic", tier: "budget",
      inputCostPer1M: 0.8, outputCostPer1M: 4, cacheCostPer1M: 1,
      cacheReadPer1M: 0.08, supportsCaching: true, supportsTools: true },
    { id: "gemini-2.0-flash", provider: "gemini", tier: "free",
      inputCostPer1M: 0.1, outputCostPer1M: 0.4,
      supportsCaching: false, supportsTools: true },
  ],
  keys: {
    anthropic: process.env.ANTHROPIC_API_KEY,
    gemini: process.env.GEMINI_API_KEY,
  },
  budget: {
    limit: 10,
    period: "daily",
    degradation: { warnAt: 0.7, forceAt: 0.9 },
  },
});

// Router picks the cheapest model that can handle the request
const response = await client.chat("What's the weather?", {
  system: "You are a helpful assistant.",
});

console.log(response.text);
console.log(response.meta.model);  // e.g. "gemini-2.0-flash"
console.log(response.meta.cost);   // { inputTokens: 150, totalCost: 0.0001, ... }

Router

The router scores message complexity and picks the cheapest model for the job:

import { Router } from "ai-cruz-tokens";

const router = new Router({
  tiers: {
    premium: ["analyze", "debug", "implement"],
    standard: ["summarize", "translate"],
  },
  defaultTier: "budget",
  lengthThreshold: 2000,
  toolAware: true,
});

router.scoreComplexity("analyze this code");  // "premium"
router.scoreComplexity("summarize this");     // "standard"
router.scoreComplexity("hello");              // "budget"
router.needsTools("what's the weather?");     // true

Prompt Stratifier

Split system prompts into cacheable static layers and dynamic layers:

import { PromptStratifier } from "ai-cruz-tokens";

const stratifier = new PromptStratifier({
  staticLayers: [
    { key: "personality", content: "You are a helpful assistant..." },
    { key: "tools", content: "Available tools: ..." },
  ],
  dynamicLayer: () => `Current time: ${new Date().toISOString()}`,
});

// For Anthropic: returns TextBlockParam[] with cache_control
const blocks = await stratifier.buildSystemBlocks("anthropic");
// For OpenAI/Gemini: returns a single string
const text = await stratifier.buildSystemBlocks("openai");

Budget Guard

Track spending and automatically degrade to cheaper models:

import { BudgetGuard } from "ai-cruz-tokens";

const budget = new BudgetGuard({
  limit: 10,
  period: "daily",
  degradation: { warnAt: 0.7, forceAt: 0.9 },
});

const tier = await budget.getAllowedTier();
// "premium" when under 70% of budget
// "standard" when 70-90%
// "free" when over 90%

Pre-fetch Pipeline

Bypass the LLM for deterministic tool workflows:

import { prefetch } from "ai-cruz-tokens";

const briefing = prefetch("morning-briefing")
  .fetch("calendar_list_events", { days: 1 }, "Calendar")
  .fetch("canvas_get_upcoming", { days: 3 }, "Assignments")
  .fetchIf(() => new Date().getDay() === 1, "expense_summary", {}, "Spending")
  .earlyExit((sections) =>
    Object.values(sections).every((s) => s.includes("nothing"))
  )
  .format("gemini-2.0-flash", "Format as a concise morning briefing.");

const result = await briefing.run(myToolExecutor, myLLMCall);

History Optimizer

Trim conversation history while preserving tool call/result pairs:

import { HistoryOptimizer } from "ai-cruz-tokens";

const optimizer = new HistoryOptimizer(20);
const trimmed = optimizer.trim(messages);
// Drops oldest messages, but never splits a tool_use from its tool_result

Built-in Pricing

Ships with current pricing for ~15 popular models. Override or extend:

import { DEFAULT_MODELS, estimateCost } from "ai-cruz-tokens";

// Estimate cost for a request
const model = DEFAULT_MODELS.find(m => m.id === "claude-sonnet-4")!;
const cost = estimateCost(model, 1000, 500); // input tokens, output tokens

License

MIT