🛡️ TokenShield & Context++

Universal AI Coding Token Economizer, Model Fatigue Shield & Context Carry-Over Suite
Cut your Cursor, Claude Code, Windsurf & Aider token bills by 70–90% while preventing AI hallucination across 4,000+ file codebases.

📦 What is TokenShield & Context++?

Modern AI coding agents (Cursor, Claude Code, Windsurf, Aider) suffer from two massive problems:

1. Model Fatigue & Context Amnesia: When codebases grow, agents blindly reload 50KB–200KB source files into context on every turn. They lose track of earlier architectural decisions and start hallucinating broken imports.
2. Exploding Cloud Token Bills: Reading whole files just to inspect a 2-line function signature costs $0.10–$0.50 per query, quickly generating hundreds of dollars in monthly API bills.

TokenShield solves this at the root by using the hardware you already paid for:

• Hybrid Dense + Sparse RAG (Light RAG): Fuses sub-3ms in-memory SQLite FTS5 BM25 for exact symbol lookups with Dense Semantic Vector Estimation (local BGE-M3 or Qwen embeddings in the execution path via LM Studio/Ollama) and a zero-dependency In-RAM Redis TCP cache (< 0.1ms). When agents ask for callers, types, or conceptual logic, TokenShield returns exact 5-line snippets in under 15ms without sending 50KB files to cloud LLMs.
• Hardware-First Interception: Your workstation has 32GB–64GB+ RAM, fast multi-core CPUs, and an idle GPU. TokenShield acts as an invisible context firewall between your editor (Cursor, Windsurf, VS Code) and cloud APIs (Claude, OpenAI) by routing code discovery and AST indexing through local hardware.
• Autonomic Local Model Manager: Probes local AI engines (Ollama on port 11434, LM Studio on port 1234). Runs lightweight coder models like qwen2.5-coder:1.5b completely offline at $0 cloud cost to catch syntax errors, missing brackets, and typos before uploading.
• Low-RAM Mode & Mixed Pipeline: Have a lightweight laptop with 8–16GB RAM? Turn off the local LLM code checker with 1 command (npx tokenshield low-ram on) while keeping ultra-fast In-RAM AST RAG active! Automatically upgrades to dense semantic search and local GPU auditing when extra hardware is detected.
• Caveman to Modern-Man (JIT Dual-Engine Transformer): Slashes expensive cloud output tokens by 60%–80% using Julius Brussee's canonical Caveman prompt to generate ultra-terse code reasoning, then auto-transforms Just-In-Time into articulate, professional Modern-Man developer walkthroughs on your local GPU/CPU (Ollama / Qwen2.5-Coder / LM Studio) for $0.00 cloud cost.
• Chat Context Carry-Over: Snapshot active tasks, touched files, and architectural decisions into a stack (chat push / chat pop), or export portable bundles (chat export / chat import) to pick up across conversations and machines without context loss.
• Free Evaluation Mode (Try Before You Buy): TokenShield operates out of the box in free evaluation mode right inside your editor so you can verify AST queries and real-time token savings before unlocking perpetual lifetime updates.
• Where Telemetry is Shown: Real-time token counts and dollar savings in your CLI, bottom VS Code status bar widget (🛡️ TokenShield: 167k saved ($3.35)), local JSON vault, and verified web pass.

🔒 Zero-PII Privacy & Local-Only Guarantee

TokenShield runs 100% locally on your machine.

• Zero Source Code Leaks: Your proprietary source code, AST call-graphs, SQLite RAM databases, prompt text, and git diffs never leave your local hardware.
• No Remote Telemetry Tracking: Detailed tool invocation traces, local logs, and file paths are stored strictly on your local disk in .agents/telemetry/.
• Anonymous Global Community Counter: The only outbound network ping is an anonymous, lightweight counter sent to praveenojha.com to power the live public community token savings ticker:

  { "tokens_saved": 48200, "ide": "vscode" }

  Strictly an aggregate numeric token count and the IDE client name are transmitted. No user IDs, no IP logging, and no project metadata.

💡 Proven in Production: Why We Built TokenShield

TokenShield was not created in a vacuum or as a theoretical demo. It was forged in production battle across 4,400+ files and 4 interconnected workspaces while architecting mission-critical, modern applications:

| Production App | Category & Technology | Live Access & Architecture | |---|---|---| | PortSync | Unified Media Distribution & Creative Automation | praveenojha.com/portsync • Multi-service REST & GPU workers | | Thermal Camera FX | Real-Time Thermal Vision & IR Simulation | Multi-spectrum thermal shaders, color palette mapping | | Animal Vision | Multi-Species Chromatic Perception Simulator | Compound eye simulation, UV spectrum emulation |

The Problem We Solved:

1. Severe Model Context Dilution: Frontier models (Claude 3.7/Sonnet, GPT-4o, Cursor Agent) dilute attention past 20–30 turns across large codebases, breaking existing conventions and hallucinating imports.
2. Exponential Cloud API Costs: Repeatedly dumping full files for 5-line edits racks up hundreds of dollars in cloud bills per week.
3. Session Amnesia: Coding agents would lose track of earlier changes, causing circular loops.

The TokenShield Solution:

By moving AST call-graph indexing into local RAM (< 15ms SQLite BM25), pre-flighting syntax on local models at $0 cost, and enforcing surgical 3-line diffs via bundled skills, our agents became laser-focused and architectural disciplined — saving 84% on cloud tokens and eliminating context fatigue.

🎨 Standalone Assets & Icons

TokenShield uses standalone, cropped individual PNG icons:

| Asset | Preview | Purpose | |---|---|---| | TokenShield Emblem | assets/icon.png | Main product shield emblem with neon emerald core | | AST Tree Graph | assets/tokenshield_ast.png | In-RAM Call-Graph and SQLite BM25 indexing engine | | Lightning Token Meter | assets/tokenshield_meter.png | Real-time token economizer and dollar cost tracker | | Terminal Pulse | assets/tokenshield_terminal.png | Context carry-over and CLI command prompt | | Accelerated Chip | assets/tokenshield_chip.png | Local GPU/CPU offline model pre-flight validator |

⚙️ System Dependencies & Autonomous Installation

TokenShield is engineered for zero manual configuration:

| Dependency | Purpose | How TokenShield Handles It | |---|---|---| | Python 3 | Core AST parser & SQLite BM25 | Auto-Detected: Uses your existing python3 or python in PATH. Never duplicates if already present. | | Node.js | Developer CLI & MCP server | Auto-Detected: Uses your existing node in PATH. | | Ollama | Offline GPU/CPU code audits ($0) | Auto-Installed: Run npx tokenshield install-ollama (or curl 1-liner) if not found. | | Docker & Redis | Sub-0.99ms In-RAM RAM cache | Auto-Provisioned: Starts Docker container tokenshield-redis or native redis-server. If absent, falls back seamlessly to built-in SQLite BM25 in-RAM mode (<2ms) with zero external dependencies required! |

💻 Universal Multi-IDE Installation Guide

TokenShield operates across all major AI coding IDEs, terminals, and agents:

| IDE / Environment | Installation Method | Configuration Target | |---|---|---| | VS Code | Install from VS Code Marketplace or code --install-extension tokenshield.vsix | Extension Sidebar & Status Bar | | Cursor | Search TokenShield in Extensions (Open-VSX) or cursor --install-extension tokenshield.vsix | .cursor/mcp.json & .cursorrules | | Windsurf (Codeium) | Search TokenShield in Extensions (Open-VSX) or windsurf --install-extension tokenshield.vsix | .codeium/windsurf/mcp_config.json & .windsurfrules | | Claude Desktop | Run npx tokenshield setup-mcp | claude_desktop_config.json | | Claude Code (CLI) | Run npx tokenshield setup-mcp | ~/.claude.json & CLAUDE.md | | Gemini / Antigravity Agent | Run npx tokenshield setup-mcp | .agents/mcp_config.json & .agents/rules/tokenshield.md | | JetBrains (IntelliJ, WebStorm, PyCharm) | Configure Model Context Protocol (MCP) using npx tokenshield mcp | JetBrains Stdio MCP Bridge | | Terminal / Neovim / Zed / Aider | Run npx tokenshield self-install or launch Cockpit: npx tokenshield tui | Interactive Terminal UI (TUI) |

⚡ 1-Command Universal Auto-Registration

Run this single command in any workspace to register TokenShield across all detected IDEs:

npx tokenshield setup-mcp

This automatically scans your system and configures VS Code, Cursor, Windsurf, Claude Desktop, Claude Code, and Gemini/Antigravity in < 50ms!

🚀 Quick Start (1 Minute)

0. ⚡ Instant 1-Click Context & Token Saver (Zero Configuration)

Arm the entire suite across all IDEs and models in < 50ms:

npx tokenshield 1click
# or:
tokenshield arm

1-Click in VS Code & Cursor:

• Bottom Status Bar: Single-click $(circle-slash) TokenShield: OFF to instantly arm 1-click protection.
• Extension Dashboard: Click [ ⚡ Instant Protect ] in the header or toggle the master card.
• Command Palette: Run TokenShield: 1-Click Token & Context Saver (Protect Now) (Ctrl+Shift+P / Cmd+Shift+P).
• Zero Modal Dialogs: All multi-IDE MCP servers are pre-configured with autoApprove: true to eliminate permission popups.

1. Install & Arm Your Project

# In your project repository:
npx tokenshield-contextplus init
# or if installed globally:
tokenshield init

This automatically indexes your code into .agents/.rag_memory.db, configures .agents/mcp_config.json, and injects context-saving rules into .cursorrules.

2. Verify Hardware & Model Status

npx tokenshield status
npx tokenshield models

3. Search Symbols with Zero Token Bloat (<15ms)

npx tokenshield ref getStoreOrder

4. Offline Pre-Flight Linting ($0.00 Cloud Cost)

npx tokenshield lint src/app/page.tsx

🎛️ Modular Component Controls (Low-RAM Optimization)

Don't have enough RAM or GPU VRAM for local LLMs? Or only need specific components? Customize your setup anytime:

# View active module states
npx tokenshield config

# 🛡️ Token Economizer Triad (Granular Runtime Controls):
npx tokenshield input on/off/status        # Input Context Guard (70–90% input tokens saved)
npx tokenshield output on/off/status       # Output Economizer (Caveman draft: 60–80% output tokens saved)
npx tokenshield interpreter on/off         # Modern-Man JIT Output Interpreter ($0 local model decompression)

# 1-Click Low-RAM Mode (disables heavy local LLMs, saves 8-14 GB RAM):
npx tokenshield low-ram on

# Turn off specific modules:
npx tokenshield disable code_checker    # Disables local LLM; uses instant zero-RAM syntax linter
npx tokenshield disable ast_rag         # Disables In-RAM AST cache

# Turn modules back on:
npx tokenshield enable code_checker
npx tokenshield enable ast_rag

🔄 Chat Context Carry-Over (push, pop, export, import)

Never let your AI coding agent forget in-flight work:

# Push current active task & touched files into stash stack:
npx tokenshield chat push "before-auth-refactor"

# List all saved chat checkpoints:
npx tokenshield chat list

# Restore a previous context by #ID:
npx tokenshield chat load 1

# Pop stashed context back into active RAM:
npx tokenshield chat pop

# Export context into a portable JSON bundle for another machine:
npx tokenshield chat export my-feature-context.json

# Import context bundle on another machine:
npx tokenshield chat import my-feature-context.json

🦖➡️👔 Caveman to Modern-Man: Just-In-Time (JIT) Dual-Engine Transformer

Output tokens on frontier models (Claude 3.7 Sonnet @ $15/1M, GPT-4o @ $10/1M, OpenAI o1 @ $60/1M) are 3x to 5x more expensive than input tokens. TokenShield delivers a revolutionary dual-engine architecture called Caveman to Modern-Man that cuts output token costs by 60%–80%:

[ Cloud Coding AI (Claude / GPT-4o) ]
       │ 
       │ 1. Phase 1: High-Density Caveman Draft (60–80% fewer output tokens)
       ▼
[ Just-In-Time (JIT) Interceptor ]
       │ 
       │ 2. Captures telegraphic code notes
       ▼
[ Local Hardware Transformer (RTX GPU / Ollama Qwen2.5-Coder / LM Studio) ]
       │ 
       │ 3. Phase 2: Decompresses JIT into articulate Modern-Man Developer Prose
       ▼
[ Professional Modern-Man Technical Walkthrough ($0.00 Cloud Cost • Zero Waste) ]

Part 1: Cloud AI Speaks High-Density Caveman

Activate Julius Brussee's canonical prompt specification across your IDE rules (.cursorrules, .windsurfrules, CLAUDE.md, .github/copilot-instructions.md):

# Enable Caveman Output Economizer across all tools:
npx tokenshield caveman on

# Check Caveman status & estimated savings:
npx tokenshield caveman

The cloud LLM strips conversational fluff and pleasantries while keeping 100% of technical substance, algorithms, and code diffs intact.

Part 2: Local Hardware Transforms into "Modern-Man" Just-In-Time ($0.00)

When teammates, PR reviewers, or clients need a full, articulate, human-friendly explanation of a terse Caveman response, TokenShield auto-transforms it Just-In-Time (JIT):

# Transform any Caveman summary or notes into an articulate Modern-Man walkthrough:
npx tokenshield modernman "fix auth header leak. wrap in timingSafeEqual. return 401 when invalid"

# Shortcuts and aliases:
npx tokenshield c2m "fix auth header leak"
npx tokenshield interpret --file terse_summary.txt

# Or invoke via MCP tool directly inside AI chat:
tokenshield_interpret_caveman({ text: "..." })
tokenshield_caveman_to_modernman({ text: "..." })

• Local Neural Model (Ollama / LM Studio): Uses local qwen2.5-coder:1.5b to expand into structured, professional documentation with verified code examples in under 1 second!
• Zero-Cloud Fallback: If offline or on a low-spec device, TokenShield's deterministic grammar engine expands abbreviations, action verbs, and clauses at $0.00 cost with 0 RAM bloat.

📊 Where Telemetry Is Displayed

1. Terminal / CLI: Run npx tokenshield telemetry to view instant latency, tool runs, tokens saved, and estimated USD prevented.
2. VS Code / Cursor Status Bar: Displays live bottom bar badge: 🛡️ TokenShield: 167k saved ($3.35) with real-time updates.
3. Local Encrypted Vault File: Persisted on disk in ~/.tokenshield/vault.json and .agents/telemetry/tool_invocations.json for offline auditability.
4. Web Portal & Printable PDF: Each purchase receives an official cryptographic PDF activation pass and instant web unlock link.

🔐 Entitlement Tiers: Free Trial vs. Pro Lifetime

| Feature | Free Community Trial | Personal Indie Pro ($29 Lifetime) | Team Agency Pro ($89 Lifetime) | |---|---|---|---| | Token Savings Limit | 100,000 tokens trial (100k) | UNLIMITED Lifetime | UNLIMITED Lifetime | | Chat Stashes (push/pop) | Max 3 checkpoints | UNLIMITED Stashes | UNLIMITED Stashes | | Context Export/Import | 2 transfers trial | UNLIMITED Transfers | UNLIMITED Transfers | | Local GPU Model Audits | Capped at 25 runs | UNLIMITED Offline Runs | UNLIMITED Offline Runs | | In-RAM AST Text RAG | Single repo | All Local Repositories | Monorepos & Workspaces | | License Seats & Devices | 1 device (7-day trial) | 1 Seat (3 devices: Desktop/Laptop/Mac) | 5 Seats (3 devices/seat = 15 total devices) |

How to Activate / Deactivate:

# Activate this machine
npx tokenshield activate <your-license-key>

# Deactivate to free up seat for another machine
npx tokenshield deactivate

Device Rotation (FIFO): If you activate on a 4th device (or 6th for Agency), the oldest device activation is automatically rotated out. You can also run npx tokenshield deactivate to explicitly free up a seat.

Purchase official lifetime keys at: https://praveenojha.com/store/tokenshield

📋 Complete CLI Command Reference

| Command | Description | |---|---| | npx tokenshield init | Arms current workspace with In-RAM AST RAG & MCP server | | npx tokenshield status | Displays active model, quota, tokens saved & cost savings | | npx tokenshield ref <sym> | Sub-15ms symbol reference & signature lookup from RAM | | npx tokenshield lint <file> | Offline local pre-flight syntax & runtime check ($0) | | npx tokenshield config | Displays and manages modular component toggles | | npx tokenshield low-ram [on\|off] | 1-click low-RAM mode (disables heavy local LLM models) | | npx tokenshield disable <mod> | Turn off component (code_checker, ast_rag, etc.) | | npx tokenshield enable <mod> | Turn on component | | npx tokenshield chat list | List all saved chat contexts & active tasks | | npx tokenshield chat save [name] | Save current chat context snapshot | | npx tokenshield chat load <id> | Load historical chat context by #ID or name | | npx tokenshield chat push [tag] | Push active context into stash stack | | npx tokenshield chat pop | Pop stashed context back into active RAM | | npx tokenshield chat export [f] | Export portable context JSON bundle | | npx tokenshield chat import <f> | Import context bundle into active session | | npx tokenshield models | Audit local hardware (probes Ollama & LM Studio) | | npx tokenshield install-ollama | Auto-installs Ollama local AI runner | | npx tokenshield pull [model] | Pulls offline coder model (e.g. qwen2.5-coder:1.5b) | | npx tokenshield diff | Surgical unified diff extractor (84% token reduction) | | npx tokenshield test | Runs the full 33-test verification suite on your machine in < 10s | | npx tokenshield activate <key> | Unlocks unlimited Lifetime Pro license | | npx tokenshield deactivate | Frees up active seat for another machine |

🧪 Verifying Your Installation (Built-in Test Suite)

TokenShield includes a 100% self-contained, zero-external-dependency automated test suite that validates all features directly on your system.

How to Run:

# In any project or terminal with npx:
npx tokenshield test

# Or if you cloned the repository:
npm test

What the Test Suite Verifies (33 Automated Tests in < 10s):

1. In-RAM Dense RAG & Folder Sculpting (rag-sculpting.test.js):
   • Dynamic folder registration (add-folder, remove-folder)
   • AST symbol indexing & sub-15ms FTS5 BM25 search
2. Chat Context Stash Manager (chat-push-pop.test.js):
   • Pushing active goals and touched files to snapshot stack (chat push)
   • Restoring sessions cleanly across context boundaries (chat pop)
3. Bundled Skills & Caveman Economizer (skills-and-caveman.test.js):
   • Verifies all bundled skills (caveman, caveman-interpret, surgical-diff, path-agnostic, zero-any, commit-craft)
   • Tests injecting and de-escalating the Caveman output economizer directive into IDE prompt rules with full attribution
4. Caveman to Modern-Man JIT Transformer (caveman-interpreter.test.js):
   • Tests local deterministic & neural decompression of telegraphic notes into professional Modern-Man developer walkthroughs at $0 cloud token cost Just-In-Time
5. Silent Deactivation & Clean Exit (silent-deactivation.test.js):
   • Asserts that turning TokenShield OFF (tokenshield off) cleanly and silently removes 100% of injected prompt blocks, leaving the developer's original .cursorrules and CLAUDE.md files completely intact with zero residue
6. Multimodal Audio Guard & Compactor (audio-guard-compressor.test.js):
   • VAD silence trimming and dead-air contraction on audio streams
   • Spoken transcript compaction (strips filler words like "um", "uh", "you know" while strictly preserving code semantics)
   • Multimodal token budget enforcement across Gemini, GPT-4o, and Whisper STT
7. Multi-IDE & JSON-RPC MCP Server Protocol (mcp-server.test.js, multi-ide.test.js):
   • Stdio MCP server handshake (initialize, tools/list, tools/call)
   • Configuration registration across VS Code, Cursor, Windsurf, Claude Code, and Antigravity IDE
8. Extension UI Buttons & Modules (extension-buttons.test.js, extension-modules.test.js):
   • Master power switch, low-RAM mode toggle, accelerator module toggles, and real-time model pricing calculation

🎖️ Community Credits & Attribution

The Caveman Output Economizer concept and prompting paradigm was originally created and pioneered by Julius Brussee (@juliusbrussee). Julius introduced the revolutionary concept of instructing AI models to discard conversational fluff, adopt high-density telegraphic fragments, and slash expensive output token bills.

TokenShield honors and integrates Julius Brussee's original work with the companion Caveman Interpreter (npx tokenshield interpret / MCP tool tokenshield_interpret_caveman), enabling round-trip local decompression ($0 cost) back into rich developer walkthroughs whenever needed.

© 2026 Praveen Ojha. All rights reserved. Support: [email protected]