@thesquirreltech/ai-token

Production-grade prompt token counter and multi-model LLM API cost estimator.
Accurately calculate token consumption and predict inference costs across OpenAI, Anthropic (Claude), Google Gemini, and DeepSeek before firing API requests.

Built with ❤️ by the engineering team at The Squirrel Technologies — creators of enterprise AI agents, automated LLM pipelines, and production AI software.

⚡ Why @thesquirreltech/ai-token?

As modern software transitions to agentic architectures and multi-step prompt chains, unpredictable LLM API costs and token limit overflows become critical production liabilities. Whether you are running autonomous agent loops, retrieval-augmented generation (RAG) pipelines, or batch document processing, calculating costs before invoking models is essential.

@thesquirreltech/ai-token provides both a lightning-fast CLI tool and a lightweight programmatic Node.js SDK to:

• Instantly measure prompt tokens across standard and frontier tokenizer encodings (including o200k_base and provider-specific ratio estimates).
• Compare live pricing across the latest model tiers (OpenAI GPT-6 Astra, GPT-5.6 Sol, GPT-5.2; Anthropic Claude Mythos 5.1, Opus 5, Sonnet 5; Google Gemini 3.1 Pro, Gemini 3.8 Flash; DeepSeek v4 Pro).
• Prevent API budget overruns by baking cost assertions directly into your agent routing logic and CI/CD pipelines.

Need help architecting cost-effective LLM systems or custom AI agents for your business? Visit The Squirrel Technologies for production AI engineering and consulting.

🚀 Quick Start (Zero Install)

Run instantly without installing anything using npx:

npx @thesquirreltech/ai-token "Analyze this enterprise codebase and summarize security vulnerabilities."

Or pass expected output token count:

npx @thesquirreltech/ai-token "Write a complete Next.js dashboard" --output 2000

📦 Installation

Global CLI

Install globally for fast access anywhere from your terminal via ai-token or stc:

npm install -g @thesquirreltech/ai-token

In Your Project

Add to your Node.js or TypeScript backend project:

npm install @thesquirreltech/ai-token

🖥️ Command Line Usage

Usage: ai-token|stc [options] [prompt]

Fast token & API price estimator across OpenAI, Anthropic, Gemini, and DeepSeek

Arguments:
  prompt                 Prompt string to estimate

Options:
  -f, --file <path>      Read prompt from file
  -m, --model <model>    Filter to a specific model
  -o, --output <tokens>  Expected output/generation tokens (default: 500)
  -h, --help             Display help for command

Examples

1. Quick Prompt Estimation

ai-token "Summarize the quarterly earnings report for stakeholders."

2. Estimating From a File

Estimate tokens and generation costs for long system prompts, transcripts, or datasets:

ai-token --file ./prompts/system_agent.md --output 1200

3. Filtering by Model

Target a specific frontier model:

ai-token "Explain the Byzantine Generals Problem" --model gpt-6-astra

4. UNIX Pipes & Stdin

Pipe outputs from other CLI utilities, git diffs, or curl requests:

git diff HEAD~1 | ai-token --output 800

💻 Programmatic JavaScript / TypeScript API

Import @thesquirreltech/ai-token into your Node.js services, background jobs, or agent routers.

1. Complete Multi-Model Prompt Analysis

Use analyzePrompt(text, estimatedOutputTokens) to get a structured breakdown across all supported models:

import { analyzePrompt } from "@thesquirreltech/ai-token";

const prompt = "You are an autonomous AI research analyst. Review the following market data...";
const expectedOutputTokens = 800;

const analysis = analyzePrompt(prompt, expectedOutputTokens);

console.log(analysis);
/*
Output:
{
  'gpt-6-astra': {
    provider: 'OpenAI',
    promptTokens: 18,
    estimatedOutputTokens: 800,
    totalTokens: 818,
    costEstimate: 0.011263
  },
  'claude-sonnet-5': {
    provider: 'Anthropic',
    promptTokens: 19,
    estimatedOutputTokens: 800,
    totalTokens: 819,
    costEstimate: 0.008038
  },
  'gemini-3.8-flash': {
    provider: 'Gemini',
    promptTokens: 19,
    estimatedOutputTokens: 800,
    totalTokens: 819,
    costEstimate: 0.0006438
  },
  ...
}
*/

2. Precise Token Counting

Count exact tokens for a given prompt and model encoding:

import { countTokens } from "@thesquirreltech/ai-token";

const text = "Evaluate architectural tradeoffs between microservices and modular monoliths.";
const tokens = countTokens(text, "gpt-6-astra");

console.log(`Prompt tokens: ${tokens}`);

3. Cost Estimation

Calculate exact inference pricing for a known input/output token volume:

import { estimateCost } from "@thesquirreltech/ai-token";

const cost = estimateCost(1500, 500, "claude-sonnet-5");
console.log(`Estimated Cost: $${cost.totalCost.toFixed(6)} ${cost.currency}`);
// Estimated Cost: $0.008000 USD

4. Accessing Raw Model Pricing Data

Inspect or extend the built-in pricing catalog:

import { MODEL_PRICING } from "@thesquirreltech/ai-token";

console.log(MODEL_PRICING["gemini-3.1-pro"]);
/*
{
  provider: 'Gemini',
  encoding: 'gemini_estimate',
  inputPer1M: 1.5,
  outputPer1M: 6.0,
  description: 'Native multimodal flagship with expansive context'
}
*/

📊 Supported Models & Pricing Matrix

| Model | Provider | Input ($ / 1M) | Output ($ / 1M) | Description | |---|---|---|---|---| | gpt-6-astra | OpenAI | $3.50 | $14.00 | Flagship general-purpose intelligence | | gpt-5.6-sol | OpenAI | $10.00 | $40.00 | Advanced reasoning & inference-time compute | | gpt-5.2-pro | OpenAI | $2.00 | $8.00 | Enterprise multi-step planning & agentic workflows | | gpt-5.2-mini | OpenAI | $0.10 | $0.40 | High-frequency, cost-effective routing & extraction | | claude-mythos-5.1 | Anthropic | $15.00 | $75.00 | Ultra-premium complex reasoning & systemic orchestration | | claude-opus-5 | Anthropic | $8.00 | $40.00 | Deep research & 1M long-context document analysis | | claude-sonnet-5 | Anthropic | $2.00 | $10.00 | Agentic coding workhorse with adaptive thinking | | claude-fable-5.1 | Anthropic | $0.50 | $2.50 | High-speed utility tier for rapid inference | | gemini-3.1-pro | Google | $1.50 | $6.00 | Native multimodal flagship with expansive context | | gemini-3.8-flash | Google | $0.20 | $0.80 | Lightweight, real-time multimodal intelligence | | deepseek-v4-pro | DeepSeek | $0.60 | $2.40 | Frontier open-weights architecture with 1M context | | muse-glimmer-30b | Meta | $0.25 | $0.25 | Ultra-fast distilled open-weights model |

🛠️ Production Best Practices for LLM Cost Optimization

When building production AI applications, consider the following strategies recommended by The Squirrel Technologies:

1. Model Cascading & Intelligent Routing: Direct high-volume, simple intent classification or entity extraction to lightweight models (gemini-3.8-flash, gpt-5.2-mini), reserving frontier reasoning models (gpt-6-astra, claude-opus-5) for nuanced execution.
2. Context Compression & Dynamic Pruning: Token counting helps determine when agent memory windows or conversation histories need summarization before crossing exponential pricing tiers.
3. Pre-flight Budget Verification: Reject or throttle batch jobs that exceed permissible threshold allocations before incurring vendor API charges.
4. CI/CD Token Regression Tests: Prevent developer prompt bloat by running @thesquirreltech/ai-token in automated testing suites to enforce strict prompt token budgets.

🌐 About The Squirrel Technologies

@thesquirreltech/ai-token is maintained by The Squirrel Technologies.

We help high-growth startups and visionary enterprises conceptualize, architect, and deploy production-ready AI systems:

• Autonomous Multi-Agent Workflows: Complex task planning, self-healing agents, and enterprise tool integrations.
• Custom LLM Solutions: Domain-specific fine-tuning, retrieval systems (RAG), and cost-optimized inference infrastructure.
• Full-Stack AI Product Engineering: From rapid prototyping to scalable, compliant cloud deployments.

Looking to build your next breakthrough AI product or optimize your team's LLM expenditure?
👉 Connect with us at thesquirrel.tech

📄 License

MIT © The Squirrel Technologies