token-estimate

"Will reading this fit in my budget?" — estimate the token / context-window cost of a file, directory, or string before an agent reads it, instead of reading blind and blowing the window.

estimate({ path: "src/" })
→ { kind: "dir", files: 38, est_tokens: 41200, pct_context: 20.6,
    top_files: [ { path: "generated/schema.ts", est_tokens: 9100 }, … ] }

Why

Agents read files with no idea of their size, and large files or directories silently eat the context window. This makes size a thing you can check first — and for a directory it points at the biggest offenders so you can skip or scope them.

The tool

estimate(path? | text?, top?) — returns:

• exact chars, words, lines, bytes
• est_tokens — a rough estimate (~chars/4, ±20%; not an exact tokenizer)
• pct_context — % of the context window (default 200k, set CONTEXT_WINDOW)
• for a directory: files count + top_files (largest first)

Directories skip node_modules/.git/build dirs; files over 2 MB are estimated from size without reading.

Setup

npm install
npm test

claude mcp add token-estimate -- node /abs/path/token-estimate/dist/server.js
# CLI: agent-tokens src/    |    agent-tokens bigfile.json

Honest limits

• It's an estimate, not a real tokenizer — chars/4 is a rule of thumb that's off by ±20% (more for dense code or non-English). Good for "2k or 50k?" decisions, not exact billing. chars/words/lines are exact.
• Reads file contents to count (skips the heavy dirs and >2 MB files).