@entelligentsia/forge-compress

Token-efficient compression for Forge SDLC tool outputs — store queries, entities, validation reports, and arbitrary markdown.

Forge agents read store queries, entity JSON, and markdown artifacts on every turn. Verbose outputs burn tokens fast. This library compresses those outputs into a form that preserves load-bearing facts (IDs, statuses, relationships, key changes) while dropping boilerplate, deep nesting, and repeated structure.

Install

npm install @entelligentsia/forge-compress

Requires Node ≥ 18. Ships ESM only.

Quick start

import {
  compressStoreQuery,
  compressEntity,
  compressMarkdown,
  compressValidateStore,
} from "@entelligentsia/forge-compress";

// Forge store query envelope → compact text
const compact = compressStoreQuery(rawJson, { maxTokens: 2000 });

// Task / bug / feature entity JSON → flat summary
const summary = compressEntity(entityJson, { keepSummaries: "latest" });

// Long markdown artifact → headings + key lines under a budget
const map = compressMarkdown(doc, { mode: "map", maxTokens: 800 });

// validate-store report → grouped errors/warnings, top-N per category
const report = compressValidateStore(reportJson);

What's included

Forge-aware compressors (src/forge/)

| Function | Input | Strategy | |---|---|---| | compressStoreQuery | Forge query envelope JSON | Drop traversal trace and config, keep IDs/titles/statuses/relationships, truncate excerpts | | compressEntity / compressEntityList | Entity record JSON (task/bug/feature/sprint) | Flatten phase summaries, keep latest verdicts and key changes | | compressMarkdown | Any markdown | Three modes: map (headings + signal lines), headings (TOC only), truncate (head-of-budget) | | compressValidateStore | validate-store JSON report | Group by category, show top-N per group, summarize the rest |

Generic primitives (src/)

| Function | Purpose | |---|---| | countTokens(text) | Heuristic token count (chars ÷ 3.7) | | truncateToTokenBudget(text, n) | Cut to a token budget on a newline boundary | | lightweightCleanup(text) | Collapse brace-only runs, strip blank-line repeats | | verbatimCompact(text) | Lossless whitespace and boilerplate strip | | stripAnsi(text) | Remove ANSI escape sequences | | compressIb(text) | Information-bottleneck compression driven by per-line entropy | | compressProgressive(items, budget) | Tiered budget allocation across an ordered list (recent items get exponentially more tokens) |

Design notes

• Lossy by intent. Compressors drop fields agents have proven they don't need (timestamps, hashes, deep config blobs). When you need full fidelity, read the source JSON directly.
• Token-bounded, not byte-bounded. Every public API accepts maxTokens and trims on token boundaries, not bytes.
• No model dependency. Pure functions. No network, no LLM calls — safe to run inside tool handlers.
• Stable shapes. Output is deterministic for a given input and option set, so it caches well in prompt-cache prefixes.

API reference

Types are exported alongside each function — see src/index.ts for the full surface. All compressors return string; pass JSON in as a raw string (they parse internally and fall back to the input on parse error).

Development

npm install
npm run build      # tsc → dist/
npm test           # vitest
npm run lint       # biome

Tests live in src/__tests__/.

License

MIT. See LICENSE.