@tokencompress/mcp

MCP (Model Context Protocol) server exposing glyph-compress for single-direction prompt compression (user→LLM only).

What is this?

This package allows AI agents (like Augment) to automatically compress large user prompts before sending them to LLMs, saving tokens and costs. Decompression is not needed — LLMs can read compressed glyphs natively after seeing the codebook once.

Installation

npm install -g @tokencompress/mcp

Or use with npx:

npx -y @tokencompress/mcp

Augment Configuration

Add to your Augment MCP settings (.augment/settings.json or equivalent):

{
  "mcpServers": {
    "glyph-compress": {
      "command": "npx",
      "args": ["-y", "@tokencompress/mcp"]
    }
  }
}

Available Tools

1. estimate_savings

Estimate token savings before compression. Helps decide whether compression is worthwhile.

Input:

{
  "text": "Your long prompt here...",
  "level": "standard",
  "provider": "raw",
  "include_codebook": false
}

Or for messages:

{
  "messages": [
    { "role": "user", "content": "Long message..." }
  ],
  "level": "standard",
  "provider": "raw",
  "include_codebook": true
}

Output:

{
  "originalTokens": 1000,
  "compressedTokens": 650,
  "codebookTokens": 186,
  "netCompressedTokens": 836,
  "netSavedTokens": 164,
  "netSavedPct": "16.4%",
  "shouldCompress": true,
  "reason": "Net savings: 164 tokens (16.4%)"
}

Parameters:

• text (optional): Text to estimate (mutually exclusive with messages)
• messages (optional): Messages array to estimate (mutually exclusive with text)
• level (optional): light | standard | aggressive | ultra (default: standard)
• provider (optional): raw | openai | anthropic | gemini | local (default: raw)
• include_codebook (optional): Include codebook cost in net calculation (default: false)

When to use:

• Before compressing large prompts to verify net savings
• When include_codebook=true, checks if codebook overhead is worth it
• shouldCompress=false means compression won't save tokens

2. compress_text

Compress a single text string.

Input:

{
  "text": "Your long prompt here...",
  "level": "standard",
  "provider": "raw",
  "include_codebook": false
}

Output:

{
  "compressed": "Compressed text with glyphs...",
  "stats": {
    "compressionRatio": 0.65,
    "originalLength": 1000,
    "compressedLength": 650,
    "ratio": "0.65x",
    "originalTokens": 250,
    "compressedTokens": 163,
    "codebookTokens": 0,
    "netCompressedTokens": 163,
    "netSavedTokens": 87,
    "netSavedPct": "34.8%"
  },
  "ratio": "0.65x",
  "too_short_to_compress": false
}

Parameters:

• text (required): Text to compress
• level (optional): light | standard | aggressive | ultra (default: standard)
• provider (optional): raw | openai | anthropic | gemini | local (default: raw)
• include_codebook (optional): Include codebook prompt in response (default: false)

Notes:

• Texts < 500 characters are not compressed (returned as-is with too_short_to_compress: true)
• netSavedTokens accounts for codebook cost when include_codebook=true
• Use estimate_savings first to check if compression is worthwhile

3. compress_messages

Compress an array of chat messages.

Input:

{
  "messages": [
    { "role": "user", "content": "Long message 1..." },
    { "role": "assistant", "content": "Long message 2..." }
  ],
  "level": "standard",
  "provider": "raw",
  "include_codebook": false
}

Output:

{
  "compressed_messages": [
    { "role": "user", "content": "Compressed..." },
    { "role": "assistant", "content": "Compressed..." }
  ],
  "stats": {
    "message_count": 2,
    "originalTokens": 500,
    "compressedTokens": 325,
    "codebookTokens": 0,
    "netCompressedTokens": 325,
    "netSavedTokens": 175,
    "netSavedPct": "35.0%",
    "shouldCompress": true,
    "strategy": "per-message"
  }
}

Parameters:

• messages (required): Array of {role, content} objects
• level (optional): Compression level (default: standard)
• provider (optional): Provider profile (default: raw)
• include_codebook (optional): Include codebook in response (default: false)

Notes:

• Messages with content < 500 characters are not compressed
• strategy indicates implementation: native (glyph-compress API) or per-message (fallback)
• Aggregate stats show total token savings across all messages

5. estimate_file_savings

Estimate token savings for a file before reading and compressing it. Useful to decide whether compress_file is worthwhile.

Input:

{
  "path": "src/large-file.ts",
  "level": "standard",
  "provider": "raw",
  "include_codebook": false,
  "max_bytes": 1048576
}

Output:

{
  "originalTokens": 3000,
  "compressedTokens": 1800,
  "codebookTokens": 186,
  "netCompressedTokens": 1986,
  "netSavedTokens": 1014,
  "netSavedPct": "33.8%",
  "shouldCompress": true,
  "reason": "Net savings: 1014 tokens (33.8%)",
  "fileBytes": 12000,
  "truncated": false
}

Parameters:

• path (required): File path relative to workspace root
• workspace_root (optional): Absolute workspace root path — always pass this to avoid cwd ambiguity
• level / provider / include_codebook: Same as estimate_savings
• max_bytes (optional): Max bytes to read before truncating (default: 1MB)
• allow_sensitive (optional): Allow reading .env/.pem/etc. (default: false)

6. compress_file

Read a file internally and return compressed content. Use this instead of a plain file-read tool to avoid sending the full raw content to the LLM.

Input:

{
  "path": "src/large-file.ts",
  "level": "standard",
  "provider": "raw",
  "include_codebook": false,
  "max_bytes": 1048576
}

Output:

{
  "compressed": "Compressed text with glyphs...",
  "stats": {
    "originalTokens": 3000,
    "compressedTokens": 1800,
    "codebookTokens": 0,
    "netCompressedTokens": 1800,
    "netSavedTokens": 1200,
    "netSavedPct": "40.0%"
  },
  "ratio": "1.7x",
  "source": { "path": "src/large-file.ts", "bytes": 12000, "truncated": false }
}

Parameters:

• path (required): File path relative to workspace root
• workspace_root (optional): Absolute workspace root path — always pass this
• level / provider / include_codebook / max_bytes / allow_sensitive: Same as above

Notes:

• Files < 500 characters are returned as-is with ratio: "1.0x"
• include_codebook=true adds codebook field and includes its cost in netCompressedTokens
• Do NOT use for source code you are about to edit — use compress_file_range for targeted reads

7. compress_file_range

Read a specific line range from a file and compress it. Ideal for large files where only a section is needed.

Input:

{
  "path": "src/large-file.ts",
  "start_line": 100,
  "end_line": 200,
  "level": "standard",
  "provider": "raw",
  "include_codebook": false
}

Output:

{
  "compressed": "Compressed lines 100-200...",
  "stats": { "originalTokens": 500, "compressedTokens": 300, "netSavedTokens": 200, "netSavedPct": "40.0%" },
  "ratio": "1.7x",
  "source": {
    "path": "src/large-file.ts",
    "bytes": 12000,
    "truncated": false,
    "start_line": 100,
    "end_line": 200,
    "total_lines": 450
  }
}

Parameters:

• path (required): File path relative to workspace root
• workspace_root (optional): Absolute workspace root path — always pass this
• start_line (optional): 1-based inclusive start line (default: 1)
• end_line (optional): 1-based inclusive end line (default: last line)
• level / provider / include_codebook / max_bytes / allow_sensitive: Same as above

4. get_codebook

Get the glyph codebook prompt for a given compression level.

Input:

{
  "level": "standard",
  "provider": "raw"
}

Output:

{
  "codebook": "[GLYPH PROTOCOL v0.5]\n...",
  "token_estimate": 450
}

Single-Direction Compression

Important: glyph-compress is designed for one-way compression (user→LLM). The LLM reads compressed prompts directly using the codebook. No decompression tool is provided because:

1. LLMs understand glyphs after seeing the codebook once
2. LLM responses are typically shorter and don't need compression
3. Decompression would add unnecessary complexity

What MCP Compression Can and Cannot Save

| Scenario | Saves tokens? | Recommended tool | |---|---|---| | User pastes large text in chat | ❌ No — user prompt is sent before MCP runs | compress_text (agent pre-processes) | | Agent reads a file via file tool | ✅ Yes — tool result enters context compressed | compress_file / compress_file_range | | Agent reads a large log/doc | ✅ Yes | compress_file with max_bytes | | Agent edits source code | ⚠️ Use range read — don't compress whole file | compress_file_range for context only |

Key insight: compress_file saves tokens on file tool results (the content returned to the LLM), not on the initial user prompt. For source code you are about to edit, prefer compress_file_range to read only the relevant section.

File Tool Security

The file tools (estimate_file_savings, compress_file, compress_file_range) enforce these restrictions:

Workspace root resolution order:

1. workspace_root parameter (pass E:\Projects\MyApp directly — recommended)
2. WORKSPACE_ROOT environment variable
3. PROJECT_ROOT environment variable
4. Auto-detect: walks up from cwd looking for .augment, .agent-bus, .git, package.json
5. Error: Cannot determine workspace root. Pass workspace_root parameter, or set WORKSPACE_ROOT / PROJECT_ROOT env var.

Always pass workspace_root when calling file tools — do not rely on cwd which may be wrong in MCP runtime.

Access rules:

• Paths must be within the resolved workspace root (traversal blocked)
• Directories are rejected (files only)
• node_modules/, .git/, .agent-bus/, .augment/ text files are allowed
• Sensitive files (.env, .key, .pem, .cert, .crt, .p12, .pfx) are blocked by default — pass allow_sensitive=true to override
• Binary files (images, audio, video, archives, executables, WASM, Office docs) are always blocked
• max_bytes (default 1MB) truncates oversized files; response includes truncated: true

Understanding Net Savings

When using compression, consider the codebook cost:

• Without codebook (include_codebook=false): Assumes LLM already has the codebook in context

  • Net savings = originalTokens - compressedTokens
  • Use for subsequent messages in the same conversation

• With codebook (include_codebook=true): Includes codebook overhead (~186 tokens for standard level)

  • Net savings = originalTokens - (compressedTokens + codebookTokens)
  • Use for the first message or standalone prompts

Recommendation:

• Use estimate_savings before compressing to check if net savings > 0
• For short prompts (<1000 tokens), codebook overhead may exceed savings
• For long prompts (>2000 tokens), compression typically saves 30-50% even with codebook

Usage with Augment Skills

See .augment/skills/glyph-compress/SKILL.md for agent instructions on when to use compression.

Trigger conditions:

• User pastes large code files (>1KB)
• User provides long documentation or logs
• Context size is approaching limits

Do NOT compress:

• .agent-bus protocol messages
• Skill files or agent instructions
• Code generation output
• Short messages (<500 chars)

Development

cd packages/mcp
npm install
npm run build
npm test

License

AGPL-3.0-only (same as glyph-compress)

Links

• glyph-compress
• Model Context Protocol