mdmin

Markdown compression + context extraction for LLM apps.

Website: mdmin.dev • npm: npmjs.com/package/mdmin • PyPI: pypi.org/project/mdmin • VS Code: marketplace

What it does

Three things, used together or independently:

1. Compress markdown — rule-based engine strips verbose phrases, redundant formatting, and structural waste. 13–35% token savings. Free, instant, zero API calls.

2. Extract relevant context — given a large document + a query, returns only the relevant chunks within a token budget. TF-IDF based, no external API, runs in milliseconds. Achieves 70–95% reduction on targeted queries. Free.

3. Manage context windows — ContextBudget automatically handles sliding message windows for chat apps, agents, and RAG pipelines. Compresses old turns before dropping them, pins critical facts that never disappear.

Install

npm install -g mdmin

pip install mdmin

CLI

Compress

mdmin compress README.md                        # output to stdout
mdmin compress README.md -o README.min.md       # save to file
mdmin compress ./docs/ --level aggressive       # batch compress directory
mdmin stats README.md                           # token stats at all levels
cat file.md | mdmin compress -                  # pipe from stdin

Extract

Extract only the sections of a document relevant to your query — no LLM required.

# Single file
mdmin extract CLAUDE.md --query "how does auth work"
mdmin extract bigdoc.md -q "database schema" --max 1500

# Query across a directory (multi-doc)
mdmin extract ./docs/ -q "authentication"
mdmin extract ./docs/ -q "auth" --recursive --max 4000

Example output:

Querying 12 files: "authentication"
Found relevant content in 3/12 files | 847 tokens

── api-docs.md ──
## Authentication
JWT tokens required. Pass Bearer token in Authorization header...

── architecture.md ──
## Auth Service
Node.js on port 3001. Handles all auth for the platform...

Programmatic

Node.js — Compress

const { compress, estimateTokens } = require('mdmin')

const { output, stats } = compress(markdownText, { level: 'medium' })
console.log(stats)
// { inputTokens: 2273, outputTokens: 1765, saved: 508, pct: 22.3 }

Node.js — Extract

const { ContextExtractor } = require('mdmin')

const extractor = new ContextExtractor()
extractor.index(largeDocument)
const { text, stats } = extractor.extract('how does auth work', { maxTokens: 2000 })

console.log(text)           // only the relevant chunks
console.log(stats.reduction) // e.g. 91.2 (%)

Python — Compress + Extract

from mdmin import compress, extract

# Compress
result = compress(text, level="medium")
print(result.stats.pct)   # 22.3

# Extract
result = extract(large_doc, "how does auth work", max_tokens=2000)
print(result.text)              # relevant chunks only
print(result.stats.reduction)   # 91.2

mdmin compress file.md --level aggressive
mdmin extract file.md -q "auth flow" --max 2000
mdmin stats file.md

Compression Levels

| Level | Savings | What it does | |---|---|---| | light | ~10% | Whitespace, comments, basic verbose patterns | | medium | ~20-25% | + more verbose patterns, table compression, formatting cleanup | | aggressive | ~25-35% | + article stripping, list compression, bold removal, dictionary dedup |

What Gets Compressed

• Verbose phrases: 150+ patterns — "In order to" → "To", "Due to the fact that" → "Because"
• Whitespace: Blank lines, trailing spaces, decorative horizontal rules
• Tables: Markdown tables → compact CSV or key:value format
• Formatting: Redundant bold on headers, deep heading nesting, emphasis markers
• Lists: Short bullet lists → inline comma-separated (aggressive)
• Links: Empty titles, unused references, verbose alt text
• Dictionary dedup: Repeated phrases replaced with §1, §2 tokens

ContextBudget — Context Window Manager

ContextBudget solves a problem every LLM app developer hits: what do you do when your conversation history exceeds the model's context window? Most developers splice off old messages with brittle ad-hoc code. ContextBudget does it properly — compressing before dropping, always protecting what matters most.

Priority order (highest to lowest)

| Priority | What | Behaviour | |----------|------|-----------| | 1st | Pins | .pin() facts are never compressed or dropped, ever | | 2nd | Recent turns | Last keepLastN messages always verbatim | | 3rd | Context docs | .addContext() docs compressed on ingestion, never dropped | | 4th | Older messages | Rule-compressed first, then dropped oldest-first if still over | | Last resort | System prompt | Only compressed if nothing else fits |

Basic usage

const { ContextBudget } = require('mdmin')

const budget = new ContextBudget({
  limit: 128_000,       // your model's context window (tokens)
  reserve: 8_000,       // headroom for LLM output
  keepLastN: 10,        // recent turns always kept verbatim
  compressionLevel: 'aggressive',
})

budget.setSystem('You are a helpful assistant.')
budget.pin('user_id=u_123')
budget.addContext(kbArticle, { label: 'Auth Guide' })

await budget.addMessage({ role: 'user', content: userInput })
const { messages, stats } = budget.get()
// Pass directly to any LLM provider

API reference

| Method | Returns | Description | |--------|---------|-------------| | .setSystem(text) | this | Set system prompt | | .addContext(text, { label }) | this | Add RAG doc — compressed on ingestion, never dropped | | .pin(fact) | this | Add a sticky fact — never dropped or compressed | | .addMessage(message) | Promise<this> | Add a chat message — triggers trim if over budget | | .get() | { messages, stats } | Get final array ready for any LLM API | | .tokenCount() | number | Current total token usage | | .clear() | this | Reset messages + contexts, keep system + pins |

MCP Server

Use mdmin as a tool in Claude Desktop, Cursor, or any MCP-compatible assistant:

{
  "mcpServers": {
    "mdmin": {
      "command": "npx",
      "args": ["mdmin-mcp"],
      "env": { "MDMIN_API_KEY": "mdmin_sk_..." }
    }
  }
}

claude mcp add mdmin npx mdmin-mcp

MCP tools: compress_markdown, extract_relevant, get_stats, manage_context_budget

API

Free for all registered users. Get your key at mdmin.dev/settings.

# Compress
curl https://mdmin.dev/api/v1/compress \
  -H "Authorization: Bearer $MDMIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"markdown": "...", "level": "aggressive"}'

# Extract (free tier: 100KB doc, 2K token budget)
curl https://mdmin.dev/api/v1/extract \
  -H "Authorization: Bearer $MDMIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"document": "...", "query": "how does auth work", "max_tokens": 2000}'

Browser Extension

Compress any selected text on any webpage in 2 clicks — no copy-paste, no tab switching.

Chrome Web Store — pending review

Select text → click the mdmin icon → compressed text is ready to copy. Works offline, no account required.

VS Code Extension

Token counter in status bar + compress document/selection from command palette or right-click.

Install from VS Code Marketplace or search "mdmin" in the Extensions panel.

Pro

$8/month — Deep Compress (LLM rewriting, 50%+ savings), compression history, Pro API limits (2MB docs, 16K token budgets on extract). Details at mdmin.dev.

License

AGPL-3.0-only — see LICENSE.txt